2023-07-29 02:58:50,610 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3 2023-07-29 02:58:50,621 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-07-29 02:58:50,647 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=292, ProcessCount=169, AvailableMemoryMB=5839 2023-07-29 02:58:50,653 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-29 02:58:50,653 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a, deleteOnExit=true 2023-07-29 02:58:50,653 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-29 02:58:50,654 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/test.cache.data in system properties and HBase conf 2023-07-29 02:58:50,654 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/hadoop.tmp.dir in system properties and HBase conf 2023-07-29 02:58:50,655 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/hadoop.log.dir in system properties and HBase conf 2023-07-29 02:58:50,655 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-29 02:58:50,656 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-29 02:58:50,656 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-29 02:58:50,753 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-29 02:58:51,049 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-29 02:58:51,052 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-29 02:58:51,052 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-29 02:58:51,053 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-29 02:58:51,053 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 02:58:51,053 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-29 02:58:51,054 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-29 02:58:51,054 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 02:58:51,054 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 02:58:51,054 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-29 02:58:51,055 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/nfs.dump.dir in system properties and HBase conf 2023-07-29 02:58:51,055 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir in system properties and HBase conf 2023-07-29 02:58:51,055 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 02:58:51,055 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-29 02:58:51,056 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-29 02:58:51,455 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 02:58:51,458 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 02:58:51,858 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-29 02:58:51,985 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-29 02:58:52,000 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:58:52,028 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:58:52,054 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir/Jetty_localhost_localdomain_35759_hdfs____.5frmb1/webapp 2023-07-29 02:58:52,188 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:35759 2023-07-29 02:58:52,197 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 02:58:52,197 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 02:58:52,683 WARN [Listener at localhost.localdomain/45449] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:58:52,780 WARN [Listener at localhost.localdomain/45449] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:58:52,796 WARN [Listener at localhost.localdomain/45449] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:58:52,802 INFO [Listener at localhost.localdomain/45449] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:58:52,807 INFO [Listener at localhost.localdomain/45449] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir/Jetty_localhost_33919_datanode____.49xwpk/webapp 2023-07-29 02:58:52,881 INFO [Listener at localhost.localdomain/45449] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33919 2023-07-29 02:58:53,178 WARN [Listener at localhost.localdomain/41391] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:58:53,188 WARN [Listener at localhost.localdomain/41391] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:58:53,194 WARN [Listener at localhost.localdomain/41391] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:58:53,196 INFO [Listener at localhost.localdomain/41391] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:58:53,201 INFO [Listener at localhost.localdomain/41391] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir/Jetty_localhost_38375_datanode____.dqzf4d/webapp 2023-07-29 02:58:53,294 INFO [Listener at localhost.localdomain/41391] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38375 2023-07-29 02:58:53,311 WARN [Listener at localhost.localdomain/42217] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:58:54,205 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x89bda5c0ffc87e72: Processing first storage report for DS-0552ddb5-544d-4017-ac63-f12a1e369be2 from datanode 0200865a-ac45-4a71-b0a7-15b490195bbe 2023-07-29 02:58:54,207 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x89bda5c0ffc87e72: from storage DS-0552ddb5-544d-4017-ac63-f12a1e369be2 node DatanodeRegistration(127.0.0.1:39825, datanodeUuid=0200865a-ac45-4a71-b0a7-15b490195bbe, infoPort=45181, infoSecurePort=0, ipcPort=42217, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-29 02:58:54,207 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x94b0edf9e4590d54: Processing first storage report for DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9 from datanode 68cee595-ec76-4947-9904-ce42ab471bd3 2023-07-29 02:58:54,207 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x94b0edf9e4590d54: from storage DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9 node DatanodeRegistration(127.0.0.1:40591, datanodeUuid=68cee595-ec76-4947-9904-ce42ab471bd3, infoPort=46163, infoSecurePort=0, ipcPort=41391, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:58:54,207 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x89bda5c0ffc87e72: Processing first storage report for DS-226b371d-ed2b-44d0-982e-ef3ec4aaa68e from datanode 0200865a-ac45-4a71-b0a7-15b490195bbe 2023-07-29 02:58:54,207 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x89bda5c0ffc87e72: from storage DS-226b371d-ed2b-44d0-982e-ef3ec4aaa68e node DatanodeRegistration(127.0.0.1:39825, datanodeUuid=0200865a-ac45-4a71-b0a7-15b490195bbe, infoPort=45181, infoSecurePort=0, ipcPort=42217, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-29 02:58:54,208 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x94b0edf9e4590d54: Processing first storage report for DS-030de696-3edd-47ec-ba8d-4cc4653e55c4 from datanode 68cee595-ec76-4947-9904-ce42ab471bd3 2023-07-29 02:58:54,208 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x94b0edf9e4590d54: from storage DS-030de696-3edd-47ec-ba8d-4cc4653e55c4 node DatanodeRegistration(127.0.0.1:40591, datanodeUuid=68cee595-ec76-4947-9904-ce42ab471bd3, infoPort=46163, infoSecurePort=0, ipcPort=41391, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:58:54,236 DEBUG [Listener at localhost.localdomain/42217] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3 2023-07-29 02:58:54,299 INFO [Listener at localhost.localdomain/42217] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/zookeeper_0, clientPort=64111, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-29 02:58:54,311 INFO [Listener at localhost.localdomain/42217] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64111 2023-07-29 02:58:54,321 INFO [Listener at localhost.localdomain/42217] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:54,324 INFO [Listener at localhost.localdomain/42217] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:54,952 INFO [Listener at localhost.localdomain/42217] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c with version=8 2023-07-29 02:58:54,952 INFO [Listener at localhost.localdomain/42217] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/hbase-staging 2023-07-29 02:58:55,194 INFO [Listener at localhost.localdomain/42217] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-29 02:58:55,551 INFO [Listener at localhost.localdomain/42217] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 02:58:55,576 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:58:55,576 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 02:58:55,576 INFO [Listener at localhost.localdomain/42217] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 02:58:55,577 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:58:55,577 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 02:58:55,689 INFO [Listener at localhost.localdomain/42217] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 02:58:55,748 DEBUG [Listener at localhost.localdomain/42217] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-29 02:58:55,820 INFO [Listener at localhost.localdomain/42217] ipc.NettyRpcServer(120): Bind to /188.40.62.62:34573 2023-07-29 02:58:55,828 INFO [Listener at localhost.localdomain/42217] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:55,830 INFO [Listener at localhost.localdomain/42217] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:55,847 INFO [Listener at localhost.localdomain/42217] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34573 connecting to ZooKeeper ensemble=127.0.0.1:64111 2023-07-29 02:58:55,950 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:345730x0, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 02:58:55,956 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34573-0x101af03ed120000 connected 2023-07-29 02:58:56,048 DEBUG [Listener at localhost.localdomain/42217] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 02:58:56,051 DEBUG [Listener at localhost.localdomain/42217] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 02:58:56,059 DEBUG [Listener at localhost.localdomain/42217] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 02:58:56,068 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34573 2023-07-29 02:58:56,068 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34573 2023-07-29 02:58:56,068 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34573 2023-07-29 02:58:56,069 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34573 2023-07-29 02:58:56,069 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34573 2023-07-29 02:58:56,074 INFO [Listener at localhost.localdomain/42217] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c, hbase.cluster.distributed=false 2023-07-29 02:58:56,132 INFO [Listener at localhost.localdomain/42217] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 02:58:56,132 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:58:56,133 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 02:58:56,133 INFO [Listener at localhost.localdomain/42217] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 02:58:56,133 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:58:56,133 INFO [Listener at localhost.localdomain/42217] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 02:58:56,137 INFO [Listener at localhost.localdomain/42217] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 02:58:56,140 INFO [Listener at localhost.localdomain/42217] ipc.NettyRpcServer(120): Bind to /188.40.62.62:46827 2023-07-29 02:58:56,142 INFO [Listener at localhost.localdomain/42217] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-29 02:58:56,148 DEBUG [Listener at localhost.localdomain/42217] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-29 02:58:56,149 INFO [Listener at localhost.localdomain/42217] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:56,151 INFO [Listener at localhost.localdomain/42217] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:56,152 INFO [Listener at localhost.localdomain/42217] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46827 connecting to ZooKeeper ensemble=127.0.0.1:64111 2023-07-29 02:58:56,164 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:468270x0, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 02:58:56,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46827-0x101af03ed120001 connected 2023-07-29 02:58:56,165 DEBUG [Listener at localhost.localdomain/42217] zookeeper.ZKUtil(164): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 02:58:56,167 DEBUG [Listener at localhost.localdomain/42217] zookeeper.ZKUtil(164): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 02:58:56,169 DEBUG [Listener at localhost.localdomain/42217] zookeeper.ZKUtil(164): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 02:58:56,169 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46827 2023-07-29 02:58:56,170 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46827 2023-07-29 02:58:56,172 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46827 2023-07-29 02:58:56,173 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46827 2023-07-29 02:58:56,173 DEBUG [Listener at localhost.localdomain/42217] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46827 2023-07-29 02:58:56,176 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:56,189 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 02:58:56,191 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:56,219 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 02:58:56,219 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 02:58:56,220 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:56,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 02:58:56,223 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,34573,1690599535066 from backup master directory 2023-07-29 02:58:56,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 02:58:56,230 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:56,231 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 02:58:56,231 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 02:58:56,232 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:56,235 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-29 02:58:56,237 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-29 02:58:56,318 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/hbase.id with ID: 72658046-9d6b-4ee3-924e-9bb534ae926b 2023-07-29 02:58:56,369 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:56,388 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:56,433 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1effa947 to 127.0.0.1:64111 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 02:58:56,473 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4cb8cd93, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 02:58:56,492 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-29 02:58:56,493 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-29 02:58:56,502 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 02:58:56,531 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store-tmp 2023-07-29 02:58:56,559 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:58:56,560 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 02:58:56,560 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:58:56,560 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:58:56,560 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 02:58:56,560 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:58:56,560 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:58:56,560 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 02:58:56,562 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:56,581 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C34573%2C1690599535066, suffix=, logDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066, archiveDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/oldWALs, maxLogs=10 2023-07-29 02:58:56,609 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-29 02:58:56,609 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-29 02:58:56,610 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:58:56,639 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:58:56,640 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39825,DS-0552ddb5-544d-4017-ac63-f12a1e369be2,DISK] 2023-07-29 02:58:56,640 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK] 2023-07-29 02:58:56,646 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:58:56,707 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 2023-07-29 02:58:56,707 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39825,DS-0552ddb5-544d-4017-ac63-f12a1e369be2,DISK], DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK]] 2023-07-29 02:58:56,708 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:58:56,708 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:58:56,711 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:58:56,712 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:58:56,763 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:58:56,769 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-29 02:58:56,788 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-29 02:58:56,798 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:56,803 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:58:56,805 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:58:56,820 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:58:56,824 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:58:56,825 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=706083, jitterRate=-0.10217025876045227}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 02:58:56,825 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 02:58:56,826 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-29 02:58:56,842 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-29 02:58:56,842 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-29 02:58:56,844 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-29 02:58:56,846 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-29 02:58:56,871 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 25 msec 2023-07-29 02:58:56,871 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-29 02:58:56,892 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-29 02:58:56,896 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-29 02:58:56,916 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-29 02:58:56,919 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-29 02:58:56,921 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-29 02:58:56,925 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-29 02:58:56,929 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-29 02:58:56,964 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:56,965 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-29 02:58:56,966 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-29 02:58:56,977 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-29 02:58:56,988 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 02:58:56,988 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 02:58:56,989 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:56,989 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase16.apache.org,34573,1690599535066, sessionid=0x101af03ed120000, setting cluster-up flag (Was=false) 2023-07-29 02:58:57,013 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:57,038 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-29 02:58:57,040 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:57,063 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:57,089 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-29 02:58:57,090 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:57,093 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.hbase-snapshot/.tmp 2023-07-29 02:58:57,178 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(951): ClusterId : 72658046-9d6b-4ee3-924e-9bb534ae926b 2023-07-29 02:58:57,182 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-29 02:58:57,190 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-29 02:58:57,196 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-29 02:58:57,196 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 02:58:57,200 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,203 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690599567202 2023-07-29 02:58:57,205 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-29 02:58:57,206 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-29 02:58:57,207 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 02:58:57,208 DEBUG [RS:0;jenkins-hbase16:46827] zookeeper.ReadOnlyZKClient(139): Connect 0x0446faf7 to 127.0.0.1:64111 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 02:58:57,208 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-29 02:58:57,213 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 02:58:57,216 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-29 02:58:57,220 DEBUG [RS:0;jenkins-hbase16:46827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a81140f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 02:58:57,221 DEBUG [RS:0;jenkins-hbase16:46827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ebe9187, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 02:58:57,223 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-29 02:58:57,223 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-29 02:58:57,224 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-29 02:58:57,224 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-29 02:58:57,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,226 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-29 02:58:57,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-29 02:58:57,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-29 02:58:57,232 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-29 02:58:57,232 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-29 02:58:57,235 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599537234,5,FailOnTimeoutGroup] 2023-07-29 02:58:57,235 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599537235,5,FailOnTimeoutGroup] 2023-07-29 02:58:57,235 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,236 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-29 02:58:57,237 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,237 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,248 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 02:58:57,249 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 02:58:57,250 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c 2023-07-29 02:58:57,253 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:46827 2023-07-29 02:58:57,259 INFO [RS:0;jenkins-hbase16:46827] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-29 02:58:57,260 INFO [RS:0;jenkins-hbase16:46827] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-29 02:58:57,260 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1022): About to register with Master. 2023-07-29 02:58:57,266 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase16.apache.org,34573,1690599535066 with isa=jenkins-hbase16.apache.org/188.40.62.62:46827, startcode=1690599536131 2023-07-29 02:58:57,267 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:58:57,271 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 02:58:57,274 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/info 2023-07-29 02:58:57,275 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 02:58:57,276 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:57,276 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 02:58:57,280 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/rep_barrier 2023-07-29 02:58:57,281 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 02:58:57,282 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:57,282 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 02:58:57,284 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/table 2023-07-29 02:58:57,285 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 02:58:57,286 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:57,287 DEBUG [RS:0;jenkins-hbase16:46827] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-29 02:58:57,288 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740 2023-07-29 02:58:57,289 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740 2023-07-29 02:58:57,292 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 02:58:57,294 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 02:58:57,297 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:58:57,299 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=870501, jitterRate=0.10690021514892578}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 02:58:57,299 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 02:58:57,299 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 02:58:57,299 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 02:58:57,299 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 02:58:57,299 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 02:58:57,299 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 02:58:57,300 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 02:58:57,300 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 02:58:57,317 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 02:58:57,317 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-29 02:58:57,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-29 02:58:57,334 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:48563, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-29 02:58:57,338 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-29 02:58:57,341 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-29 02:58:57,345 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34573] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:57,359 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c 2023-07-29 02:58:57,359 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:45449 2023-07-29 02:58:57,359 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-29 02:58:57,372 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 02:58:57,373 DEBUG [RS:0;jenkins-hbase16:46827] zookeeper.ZKUtil(162): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:57,373 WARN [RS:0;jenkins-hbase16:46827] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 02:58:57,374 INFO [RS:0;jenkins-hbase16:46827] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 02:58:57,374 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:57,375 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,46827,1690599536131] 2023-07-29 02:58:57,385 DEBUG [RS:0;jenkins-hbase16:46827] zookeeper.ZKUtil(162): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:57,394 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-29 02:58:57,402 INFO [RS:0;jenkins-hbase16:46827] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-29 02:58:57,420 INFO [RS:0;jenkins-hbase16:46827] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-29 02:58:57,422 INFO [RS:0;jenkins-hbase16:46827] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-29 02:58:57,423 INFO [RS:0;jenkins-hbase16:46827] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,424 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-29 02:58:57,430 INFO [RS:0;jenkins-hbase16:46827] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,431 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,431 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,431 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,431 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,432 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,432 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 02:58:57,432 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,432 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,432 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,432 DEBUG [RS:0;jenkins-hbase16:46827] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:58:57,433 INFO [RS:0;jenkins-hbase16:46827] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,433 INFO [RS:0;jenkins-hbase16:46827] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,433 INFO [RS:0;jenkins-hbase16:46827] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,447 INFO [RS:0;jenkins-hbase16:46827] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-29 02:58:57,449 INFO [RS:0;jenkins-hbase16:46827] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,46827,1690599536131-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:57,461 INFO [RS:0;jenkins-hbase16:46827] regionserver.Replication(203): jenkins-hbase16.apache.org,46827,1690599536131 started 2023-07-29 02:58:57,461 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1637): Serving as jenkins-hbase16.apache.org,46827,1690599536131, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:46827, sessionid=0x101af03ed120001 2023-07-29 02:58:57,461 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-29 02:58:57,462 DEBUG [RS:0;jenkins-hbase16:46827] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:57,462 DEBUG [RS:0;jenkins-hbase16:46827] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,46827,1690599536131' 2023-07-29 02:58:57,462 DEBUG [RS:0;jenkins-hbase16:46827] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:58:57,462 DEBUG [RS:0;jenkins-hbase16:46827] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:58:57,463 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-29 02:58:57,463 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-29 02:58:57,463 DEBUG [RS:0;jenkins-hbase16:46827] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:57,463 DEBUG [RS:0;jenkins-hbase16:46827] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,46827,1690599536131' 2023-07-29 02:58:57,463 DEBUG [RS:0;jenkins-hbase16:46827] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-29 02:58:57,464 DEBUG [RS:0;jenkins-hbase16:46827] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-29 02:58:57,464 DEBUG [RS:0;jenkins-hbase16:46827] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-29 02:58:57,464 INFO [RS:0;jenkins-hbase16:46827] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-29 02:58:57,464 INFO [RS:0;jenkins-hbase16:46827] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-29 02:58:57,493 DEBUG [jenkins-hbase16:34573] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-29 02:58:57,496 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,46827,1690599536131, state=OPENING 2023-07-29 02:58:57,514 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-29 02:58:57,522 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:57,523 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 02:58:57,528 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,46827,1690599536131}] 2023-07-29 02:58:57,578 INFO [RS:0;jenkins-hbase16:46827] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C46827%2C1690599536131, suffix=, logDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131, archiveDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/oldWALs, maxLogs=32 2023-07-29 02:58:57,595 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39825,DS-0552ddb5-544d-4017-ac63-f12a1e369be2,DISK] 2023-07-29 02:58:57,595 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK] 2023-07-29 02:58:57,599 INFO [RS:0;jenkins-hbase16:46827] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 2023-07-29 02:58:57,599 DEBUG [RS:0;jenkins-hbase16:46827] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39825,DS-0552ddb5-544d-4017-ac63-f12a1e369be2,DISK], DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK]] 2023-07-29 02:58:57,715 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:57,716 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-29 02:58:57,719 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:55302, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-29 02:58:57,731 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-29 02:58:57,732 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 02:58:57,736 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131, archiveDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/oldWALs, maxLogs=32 2023-07-29 02:58:57,751 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK] 2023-07-29 02:58:57,752 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39825,DS-0552ddb5-544d-4017-ac63-f12a1e369be2,DISK] 2023-07-29 02:58:57,756 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta 2023-07-29 02:58:57,757 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK], DatanodeInfoWithStorage[127.0.0.1:39825,DS-0552ddb5-544d-4017-ac63-f12a1e369be2,DISK]] 2023-07-29 02:58:57,757 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:58:57,759 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-29 02:58:57,776 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-29 02:58:57,780 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-29 02:58:57,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-29 02:58:57,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:58:57,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-29 02:58:57,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-29 02:58:57,788 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 02:58:57,790 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/info 2023-07-29 02:58:57,790 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/info 2023-07-29 02:58:57,790 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 02:58:57,791 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:57,791 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 02:58:57,793 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/rep_barrier 2023-07-29 02:58:57,793 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/rep_barrier 2023-07-29 02:58:57,794 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 02:58:57,794 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:57,795 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 02:58:57,797 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/table 2023-07-29 02:58:57,797 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/table 2023-07-29 02:58:57,797 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 02:58:57,798 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:57,800 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740 2023-07-29 02:58:57,803 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740 2023-07-29 02:58:57,807 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 02:58:57,809 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 02:58:57,811 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=779251, jitterRate=-0.009131357073783875}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 02:58:57,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 02:58:57,824 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690599537709 2023-07-29 02:58:57,841 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-29 02:58:57,842 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-29 02:58:57,842 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,46827,1690599536131, state=OPEN 2023-07-29 02:58:57,852 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-29 02:58:57,853 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 02:58:57,857 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-29 02:58:57,857 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,46827,1690599536131 in 325 msec 2023-07-29 02:58:57,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-29 02:58:57,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 534 msec 2023-07-29 02:58:57,871 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 736 msec 2023-07-29 02:58:57,871 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690599537871, completionTime=-1 2023-07-29 02:58:57,872 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-07-29 02:58:57,872 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-29 02:58:57,925 DEBUG [hconnection-0x327119bc-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 02:58:57,928 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:39986, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 02:58:57,943 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-29 02:58:57,944 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690599597944 2023-07-29 02:58:57,944 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690599657944 2023-07-29 02:58:57,944 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 71 msec 2023-07-29 02:58:58,005 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,34573,1690599535066-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:58,005 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,34573,1690599535066-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:58,005 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,34573,1690599535066-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:58,007 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:34573, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:58,008 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-29 02:58:58,016 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-29 02:58:58,023 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-29 02:58:58,024 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 02:58:58,032 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-29 02:58:58,035 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-29 02:58:58,038 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-29 02:58:58,058 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,061 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25 empty. 2023-07-29 02:58:58,062 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,062 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-29 02:58:58,093 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-29 02:58:58,096 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8d895d310595bd883153ff1142fa9e25, NAME => 'hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp 2023-07-29 02:58:58,115 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:58:58,115 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 8d895d310595bd883153ff1142fa9e25, disabling compactions & flushes 2023-07-29 02:58:58,115 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:58:58,115 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:58:58,115 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. after waiting 0 ms 2023-07-29 02:58:58,115 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:58:58,115 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:58:58,115 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 8d895d310595bd883153ff1142fa9e25: 2023-07-29 02:58:58,120 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-29 02:58:58,135 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599538123"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599538123"}]},"ts":"1690599538123"} 2023-07-29 02:58:58,158 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-29 02:58:58,160 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-29 02:58:58,165 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599538160"}]},"ts":"1690599538160"} 2023-07-29 02:58:58,170 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-29 02:58:58,219 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8d895d310595bd883153ff1142fa9e25, ASSIGN}] 2023-07-29 02:58:58,225 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8d895d310595bd883153ff1142fa9e25, ASSIGN 2023-07-29 02:58:58,227 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=8d895d310595bd883153ff1142fa9e25, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,46827,1690599536131; forceNewPlan=false, retain=false 2023-07-29 02:58:58,379 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8d895d310595bd883153ff1142fa9e25, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:58,379 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599538378"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599538378"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599538378"}]},"ts":"1690599538378"} 2023-07-29 02:58:58,392 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 8d895d310595bd883153ff1142fa9e25, server=jenkins-hbase16.apache.org,46827,1690599536131}] 2023-07-29 02:58:58,559 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:58:58,560 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8d895d310595bd883153ff1142fa9e25, NAME => 'hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25.', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:58:58,562 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,562 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:58:58,562 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,563 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,566 INFO [StoreOpener-8d895d310595bd883153ff1142fa9e25-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,569 DEBUG [StoreOpener-8d895d310595bd883153ff1142fa9e25-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/info 2023-07-29 02:58:58,570 DEBUG [StoreOpener-8d895d310595bd883153ff1142fa9e25-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/info 2023-07-29 02:58:58,570 INFO [StoreOpener-8d895d310595bd883153ff1142fa9e25-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8d895d310595bd883153ff1142fa9e25 columnFamilyName info 2023-07-29 02:58:58,572 INFO [StoreOpener-8d895d310595bd883153ff1142fa9e25-1] regionserver.HStore(310): Store=8d895d310595bd883153ff1142fa9e25/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:58:58,574 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,576 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,582 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 8d895d310595bd883153ff1142fa9e25 2023-07-29 02:58:58,586 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:58:58,587 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 8d895d310595bd883153ff1142fa9e25; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=692602, jitterRate=-0.1193120926618576}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 02:58:58,587 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 8d895d310595bd883153ff1142fa9e25: 2023-07-29 02:58:58,589 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25., pid=6, masterSystemTime=1690599538546 2023-07-29 02:58:58,592 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:58:58,593 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:58:58,594 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8d895d310595bd883153ff1142fa9e25, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:58:58,594 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599538593"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599538593"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599538593"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599538593"}]},"ts":"1690599538593"} 2023-07-29 02:58:58,602 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-29 02:58:58,602 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 8d895d310595bd883153ff1142fa9e25, server=jenkins-hbase16.apache.org,46827,1690599536131 in 206 msec 2023-07-29 02:58:58,605 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-29 02:58:58,606 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=8d895d310595bd883153ff1142fa9e25, ASSIGN in 384 msec 2023-07-29 02:58:58,607 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-29 02:58:58,607 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599538607"}]},"ts":"1690599538607"} 2023-07-29 02:58:58,610 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-29 02:58:58,649 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-29 02:58:58,650 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-29 02:58:58,655 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-29 02:58:58,655 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 625 msec 2023-07-29 02:58:58,655 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:58,695 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-29 02:58:58,722 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 02:58:58,735 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 50 msec 2023-07-29 02:58:58,742 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-29 02:58:58,763 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 02:58:58,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 34 msec 2023-07-29 02:58:58,803 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-29 02:58:58,822 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-29 02:58:58,824 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.592sec 2023-07-29 02:58:58,825 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-29 02:58:58,827 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-29 02:58:58,827 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-29 02:58:58,828 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,34573,1690599535066-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-29 02:58:58,828 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,34573,1690599535066-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-29 02:58:58,839 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-29 02:58:58,888 DEBUG [Listener at localhost.localdomain/42217] zookeeper.ReadOnlyZKClient(139): Connect 0x5e877ce3 to 127.0.0.1:64111 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 02:58:58,898 DEBUG [Listener at localhost.localdomain/42217] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3f1831dd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 02:58:58,933 DEBUG [hconnection-0x4471c172-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 02:58:58,949 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:39988, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 02:58:58,959 INFO [Listener at localhost.localdomain/42217] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:58:58,959 INFO [Listener at localhost.localdomain/42217] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:58:58,980 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-29 02:58:58,980 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:58:58,982 INFO [Listener at localhost.localdomain/42217] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-07-29 02:58:58,997 WARN [Listener at localhost.localdomain/42217] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:58:59,000 WARN [Listener at localhost.localdomain/42217] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:58:59,001 INFO [Listener at localhost.localdomain/42217] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:58:59,007 INFO [Listener at localhost.localdomain/42217] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir/Jetty_localhost_38741_datanode____.bl91yi/webapp 2023-07-29 02:58:59,085 INFO [Listener at localhost.localdomain/42217] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38741 2023-07-29 02:58:59,096 WARN [Listener at localhost.localdomain/44537] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:58:59,111 WARN [Listener at localhost.localdomain/44537] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:58:59,114 WARN [Listener at localhost.localdomain/44537] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:58:59,116 INFO [Listener at localhost.localdomain/44537] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:58:59,121 INFO [Listener at localhost.localdomain/44537] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir/Jetty_localhost_37119_datanode____.oewcx0/webapp 2023-07-29 02:58:59,194 INFO [Listener at localhost.localdomain/44537] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37119 2023-07-29 02:58:59,203 WARN [Listener at localhost.localdomain/37839] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:58:59,218 WARN [Listener at localhost.localdomain/37839] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:58:59,220 WARN [Listener at localhost.localdomain/37839] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:58:59,222 INFO [Listener at localhost.localdomain/37839] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:58:59,227 INFO [Listener at localhost.localdomain/37839] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir/Jetty_localhost_38077_datanode____.r7wutq/webapp 2023-07-29 02:58:59,305 INFO [Listener at localhost.localdomain/37839] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38077 2023-07-29 02:58:59,314 WARN [Listener at localhost.localdomain/42877] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:58:59,797 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd1dc7f1377e655fa: Processing first storage report for DS-9c59d89b-bf89-498b-b13d-073029dbfa1b from datanode ad261cba-16d0-4d51-b1bb-6d84da2352e9 2023-07-29 02:58:59,798 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd1dc7f1377e655fa: from storage DS-9c59d89b-bf89-498b-b13d-073029dbfa1b node DatanodeRegistration(127.0.0.1:44247, datanodeUuid=ad261cba-16d0-4d51-b1bb-6d84da2352e9, infoPort=35521, infoSecurePort=0, ipcPort=44537, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-29 02:58:59,798 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd1dc7f1377e655fa: Processing first storage report for DS-1669cba4-249c-4c06-96ef-ccbec6d3e08a from datanode ad261cba-16d0-4d51-b1bb-6d84da2352e9 2023-07-29 02:58:59,798 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd1dc7f1377e655fa: from storage DS-1669cba4-249c-4c06-96ef-ccbec6d3e08a node DatanodeRegistration(127.0.0.1:44247, datanodeUuid=ad261cba-16d0-4d51-b1bb-6d84da2352e9, infoPort=35521, infoSecurePort=0, ipcPort=44537, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:00,114 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x359284e0f2f95312: Processing first storage report for DS-73ef26f9-bc89-4015-b52d-98233e89a4f0 from datanode c7abc071-5cce-4ecf-bd14-978a8bc8e71a 2023-07-29 02:59:00,115 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x359284e0f2f95312: from storage DS-73ef26f9-bc89-4015-b52d-98233e89a4f0 node DatanodeRegistration(127.0.0.1:36507, datanodeUuid=c7abc071-5cce-4ecf-bd14-978a8bc8e71a, infoPort=33483, infoSecurePort=0, ipcPort=37839, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:00,115 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x359284e0f2f95312: Processing first storage report for DS-5a9fc0a3-bc82-41bd-b04e-f8abe0ad5d78 from datanode c7abc071-5cce-4ecf-bd14-978a8bc8e71a 2023-07-29 02:59:00,115 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x359284e0f2f95312: from storage DS-5a9fc0a3-bc82-41bd-b04e-f8abe0ad5d78 node DatanodeRegistration(127.0.0.1:36507, datanodeUuid=c7abc071-5cce-4ecf-bd14-978a8bc8e71a, infoPort=33483, infoSecurePort=0, ipcPort=37839, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:00,146 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x405c8ba5e482a95b: Processing first storage report for DS-123ca657-3ac3-4935-a826-43af55cf097b from datanode 2ae5c56e-722b-428e-b5f3-b33fcb25a764 2023-07-29 02:59:00,146 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x405c8ba5e482a95b: from storage DS-123ca657-3ac3-4935-a826-43af55cf097b node DatanodeRegistration(127.0.0.1:41233, datanodeUuid=2ae5c56e-722b-428e-b5f3-b33fcb25a764, infoPort=45497, infoSecurePort=0, ipcPort=42877, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:00,146 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x405c8ba5e482a95b: Processing first storage report for DS-db992ac8-4877-41dc-bb48-ec8e8b381c56 from datanode 2ae5c56e-722b-428e-b5f3-b33fcb25a764 2023-07-29 02:59:00,146 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x405c8ba5e482a95b: from storage DS-db992ac8-4877-41dc-bb48-ec8e8b381c56 node DatanodeRegistration(127.0.0.1:41233, datanodeUuid=2ae5c56e-722b-428e-b5f3-b33fcb25a764, infoPort=45497, infoSecurePort=0, ipcPort=42877, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:00,247 DEBUG [Listener at localhost.localdomain/42877] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-29 02:59:00,251 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:49674, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-29 02:59:00,259 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34573] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-07-29 02:59:00,259 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34573] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-07-29 02:59:00,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34573] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-29 02:59:00,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34573] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-07-29 02:59:00,267 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-07-29 02:59:00,269 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-29 02:59:00,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34573] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-07-29 02:59:00,272 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,273 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c empty. 2023-07-29 02:59:00,276 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,276 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-07-29 02:59:00,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34573] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-29 02:59:00,299 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-07-29 02:59:00,301 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => 15621989dd7ab4441605411cc5b22a5c, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/.tmp 2023-07-29 02:59:00,320 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:00,320 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing 15621989dd7ab4441605411cc5b22a5c, disabling compactions & flushes 2023-07-29 02:59:00,320 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:00,321 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:00,321 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. after waiting 0 ms 2023-07-29 02:59:00,321 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:00,321 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:00,321 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for 15621989dd7ab4441605411cc5b22a5c: 2023-07-29 02:59:00,325 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-07-29 02:59:00,327 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690599540327"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599540327"}]},"ts":"1690599540327"} 2023-07-29 02:59:00,330 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-29 02:59:00,332 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-29 02:59:00,332 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599540332"}]},"ts":"1690599540332"} 2023-07-29 02:59:00,335 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-07-29 02:59:00,356 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=15621989dd7ab4441605411cc5b22a5c, ASSIGN}] 2023-07-29 02:59:00,359 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=15621989dd7ab4441605411cc5b22a5c, ASSIGN 2023-07-29 02:59:00,362 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=15621989dd7ab4441605411cc5b22a5c, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,46827,1690599536131; forceNewPlan=false, retain=false 2023-07-29 02:59:00,514 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=15621989dd7ab4441605411cc5b22a5c, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:59:00,515 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690599540514"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599540514"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599540514"}]},"ts":"1690599540514"} 2023-07-29 02:59:00,521 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 15621989dd7ab4441605411cc5b22a5c, server=jenkins-hbase16.apache.org,46827,1690599536131}] 2023-07-29 02:59:00,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:00,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 15621989dd7ab4441605411cc5b22a5c, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c.', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:59:00,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath 15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:00,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,690 INFO [StoreOpener-15621989dd7ab4441605411cc5b22a5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,692 DEBUG [StoreOpener-15621989dd7ab4441605411cc5b22a5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/info 2023-07-29 02:59:00,693 DEBUG [StoreOpener-15621989dd7ab4441605411cc5b22a5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/info 2023-07-29 02:59:00,693 INFO [StoreOpener-15621989dd7ab4441605411cc5b22a5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 15621989dd7ab4441605411cc5b22a5c columnFamilyName info 2023-07-29 02:59:00,694 INFO [StoreOpener-15621989dd7ab4441605411cc5b22a5c-1] regionserver.HStore(310): Store=15621989dd7ab4441605411cc5b22a5c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:00,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:00,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:59:00,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 15621989dd7ab4441605411cc5b22a5c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=790590, jitterRate=0.0052879005670547485}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 02:59:00,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 15621989dd7ab4441605411cc5b22a5c: 2023-07-29 02:59:00,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c., pid=11, masterSystemTime=1690599540677 2023-07-29 02:59:00,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:00,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:00,709 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=15621989dd7ab4441605411cc5b22a5c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:59:00,710 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690599540709"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599540709"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599540709"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599540709"}]},"ts":"1690599540709"} 2023-07-29 02:59:00,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-29 02:59:00,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 15621989dd7ab4441605411cc5b22a5c, server=jenkins-hbase16.apache.org,46827,1690599536131 in 192 msec 2023-07-29 02:59:00,719 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-29 02:59:00,720 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=15621989dd7ab4441605411cc5b22a5c, ASSIGN in 360 msec 2023-07-29 02:59:00,721 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-29 02:59:00,721 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599540721"}]},"ts":"1690599540721"} 2023-07-29 02:59:00,723 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-07-29 02:59:00,766 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-07-29 02:59:00,770 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 504 msec 2023-07-29 02:59:03,309 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-29 02:59:03,400 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-29 02:59:03,402 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-29 02:59:03,404 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-07-29 02:59:05,190 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 02:59:05,191 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-29 02:59:10,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34573] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-29 02:59:10,290 INFO [Listener at localhost.localdomain/42877] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-07-29 02:59:10,291 DEBUG [Listener at localhost.localdomain/42877] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-07-29 02:59:10,294 INFO [Listener at localhost.localdomain/42877] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-29 02:59:10,300 INFO [Listener at localhost.localdomain/42877] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-07-29 02:59:10,301 INFO [Listener at localhost.localdomain/42877] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-29 02:59:10,301 INFO [Listener at localhost.localdomain/42877] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-07-29 02:59:10,310 DEBUG [Listener at localhost.localdomain/42877] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-07-29 02:59:10,311 DEBUG [Listener at localhost.localdomain/42877] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:10,312 WARN [Listener at localhost.localdomain/42877] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 02:59:10,320 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1555947068_17 at /127.0.0.1:43362 [Receiving block BP-993111088-188.40.62.62-1690599531514:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:40591:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:43362 dst: /127.0.0.1:40591 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,321 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-415571078_17 at /127.0.0.1:43394 [Receiving block BP-993111088-188.40.62.62-1690599531514:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:40591:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:43394 dst: /127.0.0.1:40591 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,321 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-415571078_17 at /127.0.0.1:43400 [Receiving block BP-993111088-188.40.62.62-1690599531514:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:40591:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:43400 dst: /127.0.0.1:40591 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,375 INFO [Listener at localhost.localdomain/42877] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 02:59:10,479 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-415571078_17 at /127.0.0.1:42450 [Receiving block BP-993111088-188.40.62.62-1690599531514:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:39825:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:42450 dst: /127.0.0.1:39825 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,481 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-415571078_17 at /127.0.0.1:42466 [Receiving block BP-993111088-188.40.62.62-1690599531514:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:39825:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:42466 dst: /127.0.0.1:39825 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,480 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1555947068_17 at /127.0.0.1:42422 [Receiving block BP-993111088-188.40.62.62-1690599531514:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:39825:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:42422 dst: /127.0.0.1:39825 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,484 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 02:59:10,487 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-993111088-188.40.62.62-1690599531514 (Datanode Uuid 0200865a-ac45-4a71-b0a7-15b490195bbe) service to localhost.localdomain/127.0.0.1:45449 2023-07-29 02:59:10,490 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data3/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:10,491 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data4/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:10,502 WARN [Listener at localhost.localdomain/42877] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:59:10,505 WARN [Listener at localhost.localdomain/42877] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:59:10,507 INFO [Listener at localhost.localdomain/42877] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:59:10,512 INFO [Listener at localhost.localdomain/42877] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/java.io.tmpdir/Jetty_localhost_43495_datanode____.bz0fu0/webapp 2023-07-29 02:59:10,584 INFO [Listener at localhost.localdomain/42877] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43495 2023-07-29 02:59:10,595 WARN [Listener at localhost.localdomain/36901] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:59:10,600 WARN [AsyncFSWAL-0-hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c-prefix:jenkins-hbase16.apache.org,46827,1690599536131] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,601 DEBUG [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase16.apache.org%2C46827%2C1690599536131:(num 1690599537580) roll requested 2023-07-29 02:59:10,619 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK] 2023-07-29 02:59:10,619 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36507,DS-73ef26f9-bc89-4015-b52d-98233e89a4f0,DISK] 2023-07-29 02:59:10,624 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 with entries=6, filesize=3.20 KB; new WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599550601 2023-07-29 02:59:10,624 DEBUG [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK], DatanodeInfoWithStorage[127.0.0.1:36507,DS-73ef26f9-bc89-4015-b52d-98233e89a4f0,DISK]] 2023-07-29 02:59:10,624 DEBUG [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 is not closed yet, will try archiving it next time 2023-07-29 02:59:10,626 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-29 02:59:10,626 INFO [Listener at localhost.localdomain/36901] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-29 02:59:10,628 DEBUG [Listener at localhost.localdomain/36901] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5e877ce3 to 127.0.0.1:64111 2023-07-29 02:59:10,628 DEBUG [Listener at localhost.localdomain/36901] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 02:59:10,628 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,629 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,629 DEBUG [Listener at localhost.localdomain/36901] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-29 02:59:10,629 DEBUG [Listener at localhost.localdomain/36901] util.JVMClusterUtil(257): Found active master hash=1095121702, stopped=false 2023-07-29 02:59:10,629 INFO [Listener at localhost.localdomain/36901] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:59:10,632 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 2023-07-29 02:59:10,635 WARN [IPC Server handler 2 on default port 45449] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 has not been closed. Lease recovery is in progress. RecoveryId = 1015 for block blk_1073741832_1008 2023-07-29 02:59:10,637 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 after 5ms 2023-07-29 02:59:10,662 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 02:59:10,662 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 02:59:10,662 INFO [Listener at localhost.localdomain/36901] procedure2.ProcedureExecutor(629): Stopping 2023-07-29 02:59:10,662 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:10,664 DEBUG [Listener at localhost.localdomain/36901] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1effa947 to 127.0.0.1:64111 2023-07-29 02:59:10,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 02:59:10,664 DEBUG [Listener at localhost.localdomain/36901] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 02:59:10,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 02:59:10,664 INFO [Listener at localhost.localdomain/36901] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,46827,1690599536131' ***** 2023-07-29 02:59:10,665 INFO [Listener at localhost.localdomain/36901] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-29 02:59:10,666 INFO [RS:0;jenkins-hbase16:46827] regionserver.HeapMemoryManager(220): Stopping 2023-07-29 02:59:10,666 INFO [RS:0;jenkins-hbase16:46827] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-29 02:59:10,666 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-29 02:59:10,666 INFO [RS:0;jenkins-hbase16:46827] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-29 02:59:10,667 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(3305): Received CLOSE for 8d895d310595bd883153ff1142fa9e25 2023-07-29 02:59:10,668 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(3305): Received CLOSE for 15621989dd7ab4441605411cc5b22a5c 2023-07-29 02:59:10,669 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:59:10,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 8d895d310595bd883153ff1142fa9e25, disabling compactions & flushes 2023-07-29 02:59:10,670 DEBUG [RS:0;jenkins-hbase16:46827] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0446faf7 to 127.0.0.1:64111 2023-07-29 02:59:10,670 DEBUG [RS:0;jenkins-hbase16:46827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 02:59:10,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:59:10,670 INFO [RS:0;jenkins-hbase16:46827] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-29 02:59:10,670 INFO [RS:0;jenkins-hbase16:46827] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-29 02:59:10,670 INFO [RS:0;jenkins-hbase16:46827] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-29 02:59:10,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:59:10,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. after waiting 0 ms 2023-07-29 02:59:10,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:59:10,671 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-29 02:59:10,671 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-29 02:59:10,672 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1478): Online Regions={8d895d310595bd883153ff1142fa9e25=hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25., 15621989dd7ab4441605411cc5b22a5c=TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c., 1588230740=hbase:meta,,1.1588230740} 2023-07-29 02:59:10,672 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 02:59:10,673 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 02:59:10,673 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 02:59:10,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 8d895d310595bd883153ff1142fa9e25 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-29 02:59:10,673 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 02:59:10,673 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 02:59:10,673 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.93 KB heapSize=5.45 KB 2023-07-29 02:59:10,675 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1504): Waiting on 15621989dd7ab4441605411cc5b22a5c, 1588230740, 8d895d310595bd883153ff1142fa9e25 2023-07-29 02:59:10,684 WARN [AsyncFSWAL-0-hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c-prefix:jenkins-hbase16.apache.org,46827,1690599536131.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,685 DEBUG [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta:.meta(num 1690599537737) roll requested 2023-07-29 02:59:10,701 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK] 2023-07-29 02:59:10,701 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44247,DS-9c59d89b-bf89-498b-b13d-073029dbfa1b,DISK] 2023-07-29 02:59:10,706 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599550685.meta 2023-07-29 02:59:10,706 DEBUG [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK], DatanodeInfoWithStorage[127.0.0.1:44247,DS-9c59d89b-bf89-498b-b13d-073029dbfa1b,DISK]] 2023-07-29 02:59:10,706 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,707 DEBUG [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta is not closed yet, will try archiving it next time 2023-07-29 02:59:10,708 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:10,708 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta 2023-07-29 02:59:10,709 WARN [IPC Server handler 1 on default port 45449] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta has not been closed. Lease recovery is in progress. RecoveryId = 1017 for block blk_1073741833_1009 2023-07-29 02:59:10,709 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta after 1ms 2023-07-29 02:59:10,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/.tmp/info/06621f3a6f9e4b3eb62fa296db15f304 2023-07-29 02:59:10,761 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/.tmp/info/dba1483de06f4635a7d361af5a709f75 2023-07-29 02:59:10,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/.tmp/info/06621f3a6f9e4b3eb62fa296db15f304 as hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/info/06621f3a6f9e4b3eb62fa296db15f304 2023-07-29 02:59:10,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/info/06621f3a6f9e4b3eb62fa296db15f304, entries=2, sequenceid=6, filesize=4.8 K 2023-07-29 02:59:10,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 8d895d310595bd883153ff1142fa9e25 in 155ms, sequenceid=6, compaction requested=false 2023-07-29 02:59:10,830 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/.tmp/table/0a6265253316455a8f306f088ae28051 2023-07-29 02:59:10,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/namespace/8d895d310595bd883153ff1142fa9e25/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-29 02:59:10,839 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:59:10,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 8d895d310595bd883153ff1142fa9e25: 2023-07-29 02:59:10,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690599538024.8d895d310595bd883153ff1142fa9e25. 2023-07-29 02:59:10,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 15621989dd7ab4441605411cc5b22a5c, disabling compactions & flushes 2023-07-29 02:59:10,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:10,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:10,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. after waiting 0 ms 2023-07-29 02:59:10,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:10,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 15621989dd7ab4441605411cc5b22a5c 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-07-29 02:59:10,844 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/.tmp/info/dba1483de06f4635a7d361af5a709f75 as hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/info/dba1483de06f4635a7d361af5a709f75 2023-07-29 02:59:10,857 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/info/dba1483de06f4635a7d361af5a709f75, entries=20, sequenceid=14, filesize=7.4 K 2023-07-29 02:59:10,864 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/.tmp/table/0a6265253316455a8f306f088ae28051 as hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/table/0a6265253316455a8f306f088ae28051 2023-07-29 02:59:10,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/.tmp/info/ffe5b23485c141229cb0971ee0eefe3c 2023-07-29 02:59:10,875 DEBUG [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1504): Waiting on 15621989dd7ab4441605411cc5b22a5c, 1588230740 2023-07-29 02:59:10,880 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/table/0a6265253316455a8f306f088ae28051, entries=4, sequenceid=14, filesize=4.8 K 2023-07-29 02:59:10,883 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.93 KB/2998, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 210ms, sequenceid=14, compaction requested=false 2023-07-29 02:59:10,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/.tmp/info/ffe5b23485c141229cb0971ee0eefe3c as hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/info/ffe5b23485c141229cb0971ee0eefe3c 2023-07-29 02:59:10,896 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-07-29 02:59:10,897 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 02:59:10,898 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 02:59:10,898 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 02:59:10,899 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-29 02:59:10,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/info/ffe5b23485c141229cb0971ee0eefe3c, entries=2, sequenceid=6, filesize=6.8 K 2023-07-29 02:59:10,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for 15621989dd7ab4441605411cc5b22a5c in 63ms, sequenceid=6, compaction requested=false 2023-07-29 02:59:10,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/data/default/TestLogRolling-testLogRollOnDatanodeDeath/15621989dd7ab4441605411cc5b22a5c/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-29 02:59:10,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:10,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 15621989dd7ab4441605411cc5b22a5c: 2023-07-29 02:59:10,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1690599540259.15621989dd7ab4441605411cc5b22a5c. 2023-07-29 02:59:10,929 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xaf90731b4fcd8fc9: Processing first storage report for DS-0552ddb5-544d-4017-ac63-f12a1e369be2 from datanode 0200865a-ac45-4a71-b0a7-15b490195bbe 2023-07-29 02:59:10,931 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xaf90731b4fcd8fc9: from storage DS-0552ddb5-544d-4017-ac63-f12a1e369be2 node DatanodeRegistration(127.0.0.1:44955, datanodeUuid=0200865a-ac45-4a71-b0a7-15b490195bbe, infoPort=41073, infoSecurePort=0, ipcPort=36901, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 7, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-29 02:59:10,931 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xaf90731b4fcd8fc9: Processing first storage report for DS-226b371d-ed2b-44d0-982e-ef3ec4aaa68e from datanode 0200865a-ac45-4a71-b0a7-15b490195bbe 2023-07-29 02:59:10,931 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xaf90731b4fcd8fc9: from storage DS-226b371d-ed2b-44d0-982e-ef3ec4aaa68e node DatanodeRegistration(127.0.0.1:44955, datanodeUuid=0200865a-ac45-4a71-b0a7-15b490195bbe, infoPort=41073, infoSecurePort=0, ipcPort=36901, storageInfo=lv=-57;cid=testClusterID;nsid=1943468493;c=1690599531514), blocks: 6, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:11,075 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,46827,1690599536131; all regions closed. 2023-07-29 02:59:11,434 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-29 02:59:11,446 INFO [regionserver/jenkins-hbase16:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-29 02:59:11,446 INFO [regionserver/jenkins-hbase16:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-29 02:59:14,639 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 after 4007ms 2023-07-29 02:59:14,641 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/oldWALs/jenkins-hbase16.apache.org%2C46827%2C1690599536131.1690599537580 2023-07-29 02:59:14,711 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta after 4003ms 2023-07-29 02:59:14,713 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/WALs/jenkins-hbase16.apache.org,46827,1690599536131/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/oldWALs/jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta.1690599537737.meta 2023-07-29 02:59:14,721 DEBUG [RS:0;jenkins-hbase16:46827] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/oldWALs 2023-07-29 02:59:14,721 INFO [RS:0;jenkins-hbase16:46827] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C46827%2C1690599536131.meta:.meta(num 1690599550685) 2023-07-29 02:59:14,732 DEBUG [RS:0;jenkins-hbase16:46827] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/oldWALs 2023-07-29 02:59:14,733 INFO [RS:0;jenkins-hbase16:46827] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C46827%2C1690599536131:(num 1690599550601) 2023-07-29 02:59:14,733 DEBUG [RS:0;jenkins-hbase16:46827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 02:59:14,733 INFO [RS:0;jenkins-hbase16:46827] regionserver.LeaseManager(133): Closed leases 2023-07-29 02:59:14,733 INFO [RS:0;jenkins-hbase16:46827] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-29 02:59:14,733 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 02:59:14,735 INFO [RS:0;jenkins-hbase16:46827] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:46827 2023-07-29 02:59:14,796 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 02:59:14,796 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,46827,1690599536131 2023-07-29 02:59:14,796 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 02:59:14,804 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,46827,1690599536131] 2023-07-29 02:59:14,805 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,46827,1690599536131; numProcessing=1 2023-07-29 02:59:14,813 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,46827,1690599536131 already deleted, retry=false 2023-07-29 02:59:14,813 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,46827,1690599536131 expired; onlineServers=0 2023-07-29 02:59:14,813 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,34573,1690599535066' ***** 2023-07-29 02:59:14,813 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-29 02:59:14,814 DEBUG [M:0;jenkins-hbase16:34573] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b311d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 02:59:14,814 INFO [M:0;jenkins-hbase16:34573] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,34573,1690599535066 2023-07-29 02:59:14,814 INFO [M:0;jenkins-hbase16:34573] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,34573,1690599535066; all regions closed. 2023-07-29 02:59:14,814 DEBUG [M:0;jenkins-hbase16:34573] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 02:59:14,814 DEBUG [M:0;jenkins-hbase16:34573] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-29 02:59:14,814 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-29 02:59:14,814 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599537234] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599537234,5,FailOnTimeoutGroup] 2023-07-29 02:59:14,814 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599537235] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599537235,5,FailOnTimeoutGroup] 2023-07-29 02:59:14,814 DEBUG [M:0;jenkins-hbase16:34573] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-29 02:59:14,816 INFO [M:0;jenkins-hbase16:34573] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-29 02:59:14,816 INFO [M:0;jenkins-hbase16:34573] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-29 02:59:14,816 INFO [M:0;jenkins-hbase16:34573] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-07-29 02:59:14,817 DEBUG [M:0;jenkins-hbase16:34573] master.HMaster(1512): Stopping service threads 2023-07-29 02:59:14,817 INFO [M:0;jenkins-hbase16:34573] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-29 02:59:14,817 ERROR [M:0;jenkins-hbase16:34573] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-07-29 02:59:14,818 INFO [M:0;jenkins-hbase16:34573] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-29 02:59:14,818 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-29 02:59:14,821 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-29 02:59:14,821 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:14,821 DEBUG [M:0;jenkins-hbase16:34573] zookeeper.ZKUtil(398): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-29 02:59:14,821 WARN [M:0;jenkins-hbase16:34573] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-29 02:59:14,821 INFO [M:0;jenkins-hbase16:34573] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-29 02:59:14,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 02:59:14,822 INFO [M:0;jenkins-hbase16:34573] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-29 02:59:14,823 DEBUG [M:0;jenkins-hbase16:34573] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 02:59:14,823 INFO [M:0;jenkins-hbase16:34573] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:14,823 DEBUG [M:0;jenkins-hbase16:34573] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:14,823 DEBUG [M:0;jenkins-hbase16:34573] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 02:59:14,823 DEBUG [M:0;jenkins-hbase16:34573] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:14,823 INFO [M:0;jenkins-hbase16:34573] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.13 KB heapSize=45.77 KB 2023-07-29 02:59:14,824 WARN [AsyncFSWAL-0-hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData-prefix:jenkins-hbase16.apache.org,34573,1690599535066] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:14,824 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase16.apache.org%2C34573%2C1690599535066:(num 1690599536590) roll requested 2023-07-29 02:59:14,843 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41233,DS-123ca657-3ac3-4935-a826-43af55cf097b,DISK] 2023-07-29 02:59:14,843 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK] 2023-07-29 02:59:14,849 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 with entries=89, filesize=43.98 KB; new WAL /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599554824 2023-07-29 02:59:14,850 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40591,DS-d1b12bde-c00a-4cdc-b2ee-bff67a412bf9,DISK], DatanodeInfoWithStorage[127.0.0.1:41233,DS-123ca657-3ac3-4935-a826-43af55cf097b,DISK]] 2023-07-29 02:59:14,850 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 is not closed yet, will try archiving it next time 2023-07-29 02:59:14,850 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:14,850 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-29 02:59:14,850 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 2023-07-29 02:59:14,852 WARN [IPC Server handler 4 on default port 45449] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 has not been closed. Lease recovery is in progress. RecoveryId = 1023 for block blk_1073741829_1005 2023-07-29 02:59:14,852 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 after 2ms 2023-07-29 02:59:14,869 INFO [M:0;jenkins-hbase16:34573] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.13 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b5921abe081749a7b5c11182c6af5302 2023-07-29 02:59:14,876 DEBUG [M:0;jenkins-hbase16:34573] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b5921abe081749a7b5c11182c6af5302 as hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b5921abe081749a7b5c11182c6af5302 2023-07-29 02:59:14,883 INFO [M:0;jenkins-hbase16:34573] regionserver.HStore(1080): Added hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b5921abe081749a7b5c11182c6af5302, entries=11, sequenceid=91, filesize=7.0 K 2023-07-29 02:59:14,884 INFO [M:0;jenkins-hbase16:34573] regionserver.HRegion(2948): Finished flush of dataSize ~38.13 KB/39047, heapSize ~45.75 KB/46848, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 61ms, sequenceid=91, compaction requested=false 2023-07-29 02:59:14,886 INFO [M:0;jenkins-hbase16:34573] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:14,886 DEBUG [M:0;jenkins-hbase16:34573] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 02:59:14,905 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 02:59:14,905 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): regionserver:46827-0x101af03ed120001, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 02:59:14,905 INFO [RS:0;jenkins-hbase16:46827] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,46827,1690599536131; zookeeper connection closed. 2023-07-29 02:59:14,906 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6e2adf15] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6e2adf15 2023-07-29 02:59:14,906 INFO [Listener at localhost.localdomain/36901] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-29 02:59:15,190 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 02:59:16,390 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-29 02:59:18,854 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 after 4004ms 2023-07-29 02:59:18,856 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/WALs/jenkins-hbase16.apache.org,34573,1690599535066/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/oldWALs/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 2023-07-29 02:59:18,867 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/MasterData/oldWALs/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590 to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/oldWALs/jenkins-hbase16.apache.org%2C34573%2C1690599535066.1690599536590$masterlocalwal$ 2023-07-29 02:59:18,867 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 02:59:18,867 INFO [M:0;jenkins-hbase16:34573] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-29 02:59:18,868 INFO [M:0;jenkins-hbase16:34573] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:34573 2023-07-29 02:59:18,904 DEBUG [M:0;jenkins-hbase16:34573] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,34573,1690599535066 already deleted, retry=false 2023-07-29 02:59:19,013 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 02:59:19,013 INFO [M:0;jenkins-hbase16:34573] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,34573,1690599535066; zookeeper connection closed. 2023-07-29 02:59:19,013 DEBUG [Listener at localhost.localdomain/42217-EventThread] zookeeper.ZKWatcher(600): master:34573-0x101af03ed120000, quorum=127.0.0.1:64111, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 02:59:19,015 WARN [Listener at localhost.localdomain/36901] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 02:59:19,021 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 02:59:19,126 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 02:59:19,126 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-993111088-188.40.62.62-1690599531514 (Datanode Uuid 0200865a-ac45-4a71-b0a7-15b490195bbe) service to localhost.localdomain/127.0.0.1:45449 2023-07-29 02:59:19,127 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data3/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,127 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data4/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,129 WARN [Listener at localhost.localdomain/36901] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 02:59:19,132 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 02:59:19,236 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 02:59:19,236 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-993111088-188.40.62.62-1690599531514 (Datanode Uuid 2ae5c56e-722b-428e-b5f3-b33fcb25a764) service to localhost.localdomain/127.0.0.1:45449 2023-07-29 02:59:19,237 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data9/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,237 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data10/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,238 WARN [Listener at localhost.localdomain/36901] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 02:59:19,241 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 02:59:19,345 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 02:59:19,345 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-993111088-188.40.62.62-1690599531514 (Datanode Uuid c7abc071-5cce-4ecf-bd14-978a8bc8e71a) service to localhost.localdomain/127.0.0.1:45449 2023-07-29 02:59:19,345 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data7/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,346 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data8/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,347 WARN [Listener at localhost.localdomain/36901] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 02:59:19,351 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 02:59:19,455 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 02:59:19,455 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-993111088-188.40.62.62-1690599531514 (Datanode Uuid ad261cba-16d0-4d51-b1bb-6d84da2352e9) service to localhost.localdomain/127.0.0.1:45449 2023-07-29 02:59:19,456 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data5/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,456 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data6/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,457 WARN [Listener at localhost.localdomain/36901] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 02:59:19,460 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 02:59:19,561 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 02:59:19,561 WARN [BP-993111088-188.40.62.62-1690599531514 heartbeating to localhost.localdomain/127.0.0.1:45449] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-993111088-188.40.62.62-1690599531514 (Datanode Uuid 68cee595-ec76-4947-9904-ce42ab471bd3) service to localhost.localdomain/127.0.0.1:45449 2023-07-29 02:59:19,562 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data1/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,562 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/cluster_539d9cf6-e7ff-7bee-b7b2-299ff3730c2a/dfs/data/data2/current/BP-993111088-188.40.62.62-1690599531514] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 02:59:19,589 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-07-29 02:59:19,706 INFO [Listener at localhost.localdomain/36901] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-29 02:59:19,756 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-29 02:59:19,769 INFO [Listener at localhost.localdomain/36901] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (666148691) connection to localhost.localdomain/127.0.0.1:45449 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: regionserver/jenkins-hbase16:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (666148691) connection to localhost.localdomain/127.0.0.1:45449 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase16:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost.localdomain:45449 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost.localdomain/36901 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase16:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (666148691) connection to localhost.localdomain/127.0.0.1:45449 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost.localdomain:45449 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@58ea533c java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=466 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=259 (was 292), ProcessCount=169 (was 169), AvailableMemoryMB=4823 (was 5839) 2023-07-29 02:59:19,780 INFO [Listener at localhost.localdomain/36901] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=466, MaxFileDescriptor=60000, SystemLoadAverage=259, ProcessCount=169, AvailableMemoryMB=4823 2023-07-29 02:59:19,780 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-29 02:59:19,781 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/hadoop.log.dir so I do NOT create it in target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996 2023-07-29 02:59:19,781 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/42cf9ab3-2cda-25c7-6038-95a85bce8ea3/hadoop.tmp.dir so I do NOT create it in target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996 2023-07-29 02:59:19,781 INFO [Listener at localhost.localdomain/36901] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686, deleteOnExit=true 2023-07-29 02:59:19,781 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-29 02:59:19,781 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/test.cache.data in system properties and HBase conf 2023-07-29 02:59:19,782 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/hadoop.tmp.dir in system properties and HBase conf 2023-07-29 02:59:19,782 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/hadoop.log.dir in system properties and HBase conf 2023-07-29 02:59:19,782 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-29 02:59:19,782 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-29 02:59:19,782 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-29 02:59:19,782 DEBUG [Listener at localhost.localdomain/36901] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-29 02:59:19,783 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-29 02:59:19,783 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-29 02:59:19,783 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-29 02:59:19,783 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 02:59:19,783 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-29 02:59:19,783 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-29 02:59:19,783 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 02:59:19,784 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 02:59:19,784 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-29 02:59:19,784 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/nfs.dump.dir in system properties and HBase conf 2023-07-29 02:59:19,784 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/java.io.tmpdir in system properties and HBase conf 2023-07-29 02:59:19,784 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 02:59:19,784 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-29 02:59:19,784 INFO [Listener at localhost.localdomain/36901] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-29 02:59:19,787 WARN [Listener at localhost.localdomain/36901] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 02:59:19,787 WARN [Listener at localhost.localdomain/36901] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 02:59:20,025 WARN [Listener at localhost.localdomain/36901] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:59:20,027 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:59:20,032 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/java.io.tmpdir/Jetty_localhost_localdomain_33663_hdfs____.gza3gz/webapp 2023-07-29 02:59:20,119 INFO [Listener at localhost.localdomain/36901] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:33663 2023-07-29 02:59:20,124 WARN [Listener at localhost.localdomain/36901] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 02:59:20,124 WARN [Listener at localhost.localdomain/36901] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 02:59:20,270 WARN [Listener at localhost.localdomain/40955] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:59:20,287 WARN [Listener at localhost.localdomain/40955] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:59:20,289 WARN [Listener at localhost.localdomain/40955] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:59:20,290 INFO [Listener at localhost.localdomain/40955] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:59:20,295 INFO [Listener at localhost.localdomain/40955] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/java.io.tmpdir/Jetty_localhost_38905_datanode____.iq01sw/webapp 2023-07-29 02:59:20,372 INFO [Listener at localhost.localdomain/40955] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38905 2023-07-29 02:59:20,378 WARN [Listener at localhost.localdomain/45943] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:59:20,392 WARN [Listener at localhost.localdomain/45943] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 02:59:20,395 WARN [Listener at localhost.localdomain/45943] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 02:59:20,396 INFO [Listener at localhost.localdomain/45943] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 02:59:20,400 INFO [Listener at localhost.localdomain/45943] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/java.io.tmpdir/Jetty_localhost_43505_datanode____h6egea/webapp 2023-07-29 02:59:20,483 INFO [Listener at localhost.localdomain/45943] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43505 2023-07-29 02:59:20,498 WARN [Listener at localhost.localdomain/41871] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 02:59:21,214 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4fdc0a05bf58bee7: Processing first storage report for DS-863478f6-b727-42d4-a199-c252c0df1f8d from datanode 2e608275-5f4d-4a8a-aa2d-0bc309941c30 2023-07-29 02:59:21,214 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4fdc0a05bf58bee7: from storage DS-863478f6-b727-42d4-a199-c252c0df1f8d node DatanodeRegistration(127.0.0.1:46401, datanodeUuid=2e608275-5f4d-4a8a-aa2d-0bc309941c30, infoPort=40707, infoSecurePort=0, ipcPort=45943, storageInfo=lv=-57;cid=testClusterID;nsid=506601530;c=1690599559790), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:21,214 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4fdc0a05bf58bee7: Processing first storage report for DS-925ea6a2-1c1a-444c-bff1-e222b651ddaf from datanode 2e608275-5f4d-4a8a-aa2d-0bc309941c30 2023-07-29 02:59:21,214 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4fdc0a05bf58bee7: from storage DS-925ea6a2-1c1a-444c-bff1-e222b651ddaf node DatanodeRegistration(127.0.0.1:46401, datanodeUuid=2e608275-5f4d-4a8a-aa2d-0bc309941c30, infoPort=40707, infoSecurePort=0, ipcPort=45943, storageInfo=lv=-57;cid=testClusterID;nsid=506601530;c=1690599559790), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:21,350 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x80c45187700ff5d0: Processing first storage report for DS-97cf72b3-554a-4b26-ac2d-6b48bf182471 from datanode 36071cb8-68c0-4991-8095-3d127e30f924 2023-07-29 02:59:21,350 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x80c45187700ff5d0: from storage DS-97cf72b3-554a-4b26-ac2d-6b48bf182471 node DatanodeRegistration(127.0.0.1:37773, datanodeUuid=36071cb8-68c0-4991-8095-3d127e30f924, infoPort=35751, infoSecurePort=0, ipcPort=41871, storageInfo=lv=-57;cid=testClusterID;nsid=506601530;c=1690599559790), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:21,352 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x80c45187700ff5d0: Processing first storage report for DS-623488d6-aa3a-43c8-9084-1c540780e0eb from datanode 36071cb8-68c0-4991-8095-3d127e30f924 2023-07-29 02:59:21,352 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x80c45187700ff5d0: from storage DS-623488d6-aa3a-43c8-9084-1c540780e0eb node DatanodeRegistration(127.0.0.1:37773, datanodeUuid=36071cb8-68c0-4991-8095-3d127e30f924, infoPort=35751, infoSecurePort=0, ipcPort=41871, storageInfo=lv=-57;cid=testClusterID;nsid=506601530;c=1690599559790), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 02:59:21,427 DEBUG [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996 2023-07-29 02:59:21,436 INFO [Listener at localhost.localdomain/41871] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686/zookeeper_0, clientPort=49643, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-29 02:59:21,439 INFO [Listener at localhost.localdomain/41871] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=49643 2023-07-29 02:59:21,439 INFO [Listener at localhost.localdomain/41871] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:21,440 INFO [Listener at localhost.localdomain/41871] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:21,457 INFO [Listener at localhost.localdomain/41871] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772 with version=8 2023-07-29 02:59:21,457 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/hbase-staging 2023-07-29 02:59:21,459 INFO [Listener at localhost.localdomain/41871] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 02:59:21,460 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:59:21,460 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 02:59:21,460 INFO [Listener at localhost.localdomain/41871] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 02:59:21,460 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:59:21,460 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 02:59:21,460 INFO [Listener at localhost.localdomain/41871] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 02:59:21,462 INFO [Listener at localhost.localdomain/41871] ipc.NettyRpcServer(120): Bind to /188.40.62.62:41613 2023-07-29 02:59:21,462 INFO [Listener at localhost.localdomain/41871] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:21,464 INFO [Listener at localhost.localdomain/41871] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:21,465 INFO [Listener at localhost.localdomain/41871] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41613 connecting to ZooKeeper ensemble=127.0.0.1:49643 2023-07-29 02:59:21,518 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:416130x0, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 02:59:21,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41613-0x101af0457210000 connected 2023-07-29 02:59:21,588 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 02:59:21,588 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 02:59:21,589 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 02:59:21,589 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41613 2023-07-29 02:59:21,589 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41613 2023-07-29 02:59:21,590 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41613 2023-07-29 02:59:21,590 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41613 2023-07-29 02:59:21,591 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41613 2023-07-29 02:59:21,591 INFO [Listener at localhost.localdomain/41871] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772, hbase.cluster.distributed=false 2023-07-29 02:59:21,607 INFO [Listener at localhost.localdomain/41871] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 02:59:21,607 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:59:21,608 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 02:59:21,608 INFO [Listener at localhost.localdomain/41871] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 02:59:21,608 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 02:59:21,608 INFO [Listener at localhost.localdomain/41871] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 02:59:21,608 INFO [Listener at localhost.localdomain/41871] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 02:59:21,611 INFO [Listener at localhost.localdomain/41871] ipc.NettyRpcServer(120): Bind to /188.40.62.62:41957 2023-07-29 02:59:21,611 INFO [Listener at localhost.localdomain/41871] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-29 02:59:21,614 DEBUG [Listener at localhost.localdomain/41871] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-29 02:59:21,615 INFO [Listener at localhost.localdomain/41871] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:21,617 INFO [Listener at localhost.localdomain/41871] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:21,618 INFO [Listener at localhost.localdomain/41871] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41957 connecting to ZooKeeper ensemble=127.0.0.1:49643 2023-07-29 02:59:21,629 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:419570x0, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 02:59:21,631 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ZKUtil(164): regionserver:419570x0, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 02:59:21,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41957-0x101af0457210001 connected 2023-07-29 02:59:21,632 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 02:59:21,633 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 02:59:21,635 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41957 2023-07-29 02:59:21,635 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41957 2023-07-29 02:59:21,635 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41957 2023-07-29 02:59:21,638 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41957 2023-07-29 02:59:21,638 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41957 2023-07-29 02:59:21,639 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:21,646 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 02:59:21,646 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:21,654 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 02:59:21,654 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 02:59:21,655 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:21,655 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 02:59:21,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 02:59:21,656 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,41613,1690599561459 from backup master directory 2023-07-29 02:59:21,662 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:21,662 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 02:59:21,662 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 02:59:21,663 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:21,682 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/hbase.id with ID: e235e272-65f6-49d2-b09b-743de9479cc0 2023-07-29 02:59:21,699 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:21,710 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:21,724 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7f3eafee to 127.0.0.1:49643 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 02:59:21,736 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2db41701, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 02:59:21,736 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-29 02:59:21,737 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-29 02:59:21,737 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 02:59:21,739 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store-tmp 2023-07-29 02:59:21,752 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:21,752 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 02:59:21,752 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:21,752 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:21,753 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 02:59:21,753 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:21,753 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 02:59:21,753 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 02:59:21,753 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/WALs/jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:21,757 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C41613%2C1690599561459, suffix=, logDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/WALs/jenkins-hbase16.apache.org,41613,1690599561459, archiveDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/oldWALs, maxLogs=10 2023-07-29 02:59:21,772 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK] 2023-07-29 02:59:21,774 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK] 2023-07-29 02:59:21,777 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/WALs/jenkins-hbase16.apache.org,41613,1690599561459/jenkins-hbase16.apache.org%2C41613%2C1690599561459.1690599561757 2023-07-29 02:59:21,777 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK], DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK]] 2023-07-29 02:59:21,777 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:59:21,778 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:21,778 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:59:21,778 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:59:21,780 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:59:21,782 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-29 02:59:21,783 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-29 02:59:21,784 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:21,785 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:59:21,786 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:59:21,789 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 02:59:21,792 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:59:21,793 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=877968, jitterRate=0.11639434099197388}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 02:59:21,793 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 02:59:21,794 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-29 02:59:21,795 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-29 02:59:21,795 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-29 02:59:21,795 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-29 02:59:21,796 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-29 02:59:21,797 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-29 02:59:21,797 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-29 02:59:21,798 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-29 02:59:21,800 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-29 02:59:21,810 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-29 02:59:21,810 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-29 02:59:21,811 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-29 02:59:21,811 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-29 02:59:21,811 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-29 02:59:21,821 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:21,822 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-29 02:59:21,822 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-29 02:59:21,823 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-29 02:59:21,829 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 02:59:21,829 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 02:59:21,829 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:21,830 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase16.apache.org,41613,1690599561459, sessionid=0x101af0457210000, setting cluster-up flag (Was=false) 2023-07-29 02:59:21,846 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:21,871 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-29 02:59:21,873 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:21,893 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:21,921 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-29 02:59:21,923 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:21,924 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.hbase-snapshot/.tmp 2023-07-29 02:59:21,926 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 02:59:21,927 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:21,930 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690599591930 2023-07-29 02:59:21,931 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-29 02:59:21,931 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-29 02:59:21,931 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-29 02:59:21,931 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-29 02:59:21,931 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-29 02:59:21,931 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-29 02:59:21,931 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:21,932 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-29 02:59:21,932 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-29 02:59:21,932 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-29 02:59:21,933 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 02:59:21,933 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-29 02:59:21,934 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-29 02:59:21,934 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-29 02:59:21,934 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599561934,5,FailOnTimeoutGroup] 2023-07-29 02:59:21,934 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599561934,5,FailOnTimeoutGroup] 2023-07-29 02:59:21,934 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:21,934 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-29 02:59:21,934 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:21,934 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:21,935 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 02:59:21,941 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(951): ClusterId : e235e272-65f6-49d2-b09b-743de9479cc0 2023-07-29 02:59:21,942 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-29 02:59:21,950 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 02:59:21,952 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 02:59:21,952 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772 2023-07-29 02:59:21,952 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-29 02:59:21,952 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-29 02:59:21,966 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-29 02:59:21,967 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:21,967 DEBUG [RS:0;jenkins-hbase16:41957] zookeeper.ReadOnlyZKClient(139): Connect 0x233103ae to 127.0.0.1:49643 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 02:59:21,975 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 02:59:21,978 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/info 2023-07-29 02:59:21,979 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 02:59:21,980 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:21,980 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 02:59:21,982 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/rep_barrier 2023-07-29 02:59:21,983 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 02:59:21,983 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:21,984 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 02:59:21,985 DEBUG [RS:0;jenkins-hbase16:41957] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2df6f9b8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 02:59:21,986 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/table 2023-07-29 02:59:21,986 DEBUG [RS:0;jenkins-hbase16:41957] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29c968de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 02:59:21,987 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 02:59:21,988 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:21,989 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740 2023-07-29 02:59:21,990 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740 2023-07-29 02:59:21,993 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 02:59:21,995 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 02:59:21,998 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:59:21,999 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:41957 2023-07-29 02:59:21,999 INFO [RS:0;jenkins-hbase16:41957] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-29 02:59:21,999 INFO [RS:0;jenkins-hbase16:41957] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-29 02:59:21,999 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1022): About to register with Master. 2023-07-29 02:59:21,999 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=716597, jitterRate=-0.08880099654197693}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 02:59:21,999 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase16.apache.org,41613,1690599561459 with isa=jenkins-hbase16.apache.org/188.40.62.62:41957, startcode=1690599561607 2023-07-29 02:59:22,000 DEBUG [RS:0;jenkins-hbase16:41957] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-29 02:59:22,000 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 02:59:22,000 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 02:59:22,000 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 02:59:22,000 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 02:59:22,000 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 02:59:22,000 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 02:59:22,002 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 02:59:22,002 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 02:59:22,003 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:42161, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-29 02:59:22,005 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41613] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,005 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 02:59:22,005 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-29 02:59:22,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-29 02:59:22,005 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772 2023-07-29 02:59:22,005 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:40955 2023-07-29 02:59:22,005 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-29 02:59:22,007 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-29 02:59:22,009 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-29 02:59:22,012 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 02:59:22,013 DEBUG [RS:0;jenkins-hbase16:41957] zookeeper.ZKUtil(162): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,013 WARN [RS:0;jenkins-hbase16:41957] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 02:59:22,014 INFO [RS:0;jenkins-hbase16:41957] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 02:59:22,014 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,014 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,41957,1690599561607] 2023-07-29 02:59:22,018 DEBUG [RS:0;jenkins-hbase16:41957] zookeeper.ZKUtil(162): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,020 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-29 02:59:22,021 INFO [RS:0;jenkins-hbase16:41957] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-29 02:59:22,023 INFO [RS:0;jenkins-hbase16:41957] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-29 02:59:22,023 INFO [RS:0;jenkins-hbase16:41957] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-29 02:59:22,023 INFO [RS:0;jenkins-hbase16:41957] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,025 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-29 02:59:22,026 INFO [RS:0;jenkins-hbase16:41957] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,026 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,026 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,026 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,027 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,027 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,027 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 02:59:22,027 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,027 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,027 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,027 DEBUG [RS:0;jenkins-hbase16:41957] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 02:59:22,029 INFO [RS:0;jenkins-hbase16:41957] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,029 INFO [RS:0;jenkins-hbase16:41957] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,029 INFO [RS:0;jenkins-hbase16:41957] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,041 INFO [RS:0;jenkins-hbase16:41957] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-29 02:59:22,041 INFO [RS:0;jenkins-hbase16:41957] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41957,1690599561607-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,052 INFO [RS:0;jenkins-hbase16:41957] regionserver.Replication(203): jenkins-hbase16.apache.org,41957,1690599561607 started 2023-07-29 02:59:22,052 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1637): Serving as jenkins-hbase16.apache.org,41957,1690599561607, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:41957, sessionid=0x101af0457210001 2023-07-29 02:59:22,052 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-29 02:59:22,052 DEBUG [RS:0;jenkins-hbase16:41957] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,053 DEBUG [RS:0;jenkins-hbase16:41957] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,41957,1690599561607' 2023-07-29 02:59:22,053 DEBUG [RS:0;jenkins-hbase16:41957] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:59:22,053 DEBUG [RS:0;jenkins-hbase16:41957] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:59:22,054 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-29 02:59:22,054 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-29 02:59:22,054 DEBUG [RS:0;jenkins-hbase16:41957] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,054 DEBUG [RS:0;jenkins-hbase16:41957] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,41957,1690599561607' 2023-07-29 02:59:22,054 DEBUG [RS:0;jenkins-hbase16:41957] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-29 02:59:22,054 DEBUG [RS:0;jenkins-hbase16:41957] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-29 02:59:22,054 DEBUG [RS:0;jenkins-hbase16:41957] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-29 02:59:22,054 INFO [RS:0;jenkins-hbase16:41957] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-29 02:59:22,054 INFO [RS:0;jenkins-hbase16:41957] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-29 02:59:22,157 INFO [RS:0;jenkins-hbase16:41957] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C41957%2C1690599561607, suffix=, logDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607, archiveDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/oldWALs, maxLogs=32 2023-07-29 02:59:22,160 DEBUG [jenkins-hbase16:41613] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-29 02:59:22,160 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,41957,1690599561607, state=OPENING 2023-07-29 02:59:22,171 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-29 02:59:22,182 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:22,183 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK] 2023-07-29 02:59:22,183 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,41957,1690599561607}] 2023-07-29 02:59:22,183 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 02:59:22,183 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK] 2023-07-29 02:59:22,187 INFO [RS:0;jenkins-hbase16:41957] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599562158 2023-07-29 02:59:22,190 DEBUG [RS:0;jenkins-hbase16:41957] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK], DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK]] 2023-07-29 02:59:22,338 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,338 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-29 02:59:22,341 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:54798, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-29 02:59:22,345 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-29 02:59:22,345 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 02:59:22,347 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C41957%2C1690599561607.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607, archiveDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/oldWALs, maxLogs=32 2023-07-29 02:59:22,367 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK] 2023-07-29 02:59:22,368 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK] 2023-07-29 02:59:22,371 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.meta.1690599562349.meta 2023-07-29 02:59:22,371 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK], DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK]] 2023-07-29 02:59:22,371 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:59:22,372 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-29 02:59:22,372 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-29 02:59:22,373 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-29 02:59:22,373 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-29 02:59:22,373 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:22,373 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-29 02:59:22,373 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-29 02:59:22,377 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 02:59:22,378 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/info 2023-07-29 02:59:22,378 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/info 2023-07-29 02:59:22,379 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 02:59:22,380 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:22,380 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 02:59:22,381 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/rep_barrier 2023-07-29 02:59:22,381 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/rep_barrier 2023-07-29 02:59:22,381 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 02:59:22,382 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:22,382 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 02:59:22,384 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/table 2023-07-29 02:59:22,384 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/table 2023-07-29 02:59:22,385 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 02:59:22,386 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:22,387 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740 2023-07-29 02:59:22,388 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740 2023-07-29 02:59:22,390 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 02:59:22,393 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 02:59:22,394 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=858281, jitterRate=0.09136191010475159}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 02:59:22,394 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 02:59:22,395 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690599562338 2023-07-29 02:59:22,399 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-29 02:59:22,399 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-29 02:59:22,400 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,41957,1690599561607, state=OPEN 2023-07-29 02:59:22,412 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-29 02:59:22,413 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 02:59:22,417 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-29 02:59:22,417 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,41957,1690599561607 in 229 msec 2023-07-29 02:59:22,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-29 02:59:22,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 412 msec 2023-07-29 02:59:22,423 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 496 msec 2023-07-29 02:59:22,423 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690599562423, completionTime=-1 2023-07-29 02:59:22,423 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-07-29 02:59:22,423 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-29 02:59:22,426 DEBUG [hconnection-0x55072dd6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 02:59:22,429 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:54804, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 02:59:22,430 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-29 02:59:22,431 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690599622431 2023-07-29 02:59:22,431 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690599682431 2023-07-29 02:59:22,431 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 7 msec 2023-07-29 02:59:22,455 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41613,1690599561459-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,455 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41613,1690599561459-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41613,1690599561459-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:41613, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-29 02:59:22,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-29 02:59:22,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 02:59:22,459 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-29 02:59:22,459 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-29 02:59:22,463 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-29 02:59:22,464 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-29 02:59:22,468 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,468 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb empty. 2023-07-29 02:59:22,469 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,469 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-29 02:59:22,490 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-29 02:59:22,492 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => edcc0b5dc968f7ed1ca7e4cba79b64fb, NAME => 'hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp 2023-07-29 02:59:22,510 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:22,510 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing edcc0b5dc968f7ed1ca7e4cba79b64fb, disabling compactions & flushes 2023-07-29 02:59:22,510 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:22,510 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:22,511 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. after waiting 0 ms 2023-07-29 02:59:22,511 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:22,511 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:22,511 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for edcc0b5dc968f7ed1ca7e4cba79b64fb: 2023-07-29 02:59:22,514 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-29 02:59:22,516 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599562516"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599562516"}]},"ts":"1690599562516"} 2023-07-29 02:59:22,519 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-29 02:59:22,521 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-29 02:59:22,521 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599562521"}]},"ts":"1690599562521"} 2023-07-29 02:59:22,523 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-29 02:59:22,563 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=edcc0b5dc968f7ed1ca7e4cba79b64fb, ASSIGN}] 2023-07-29 02:59:22,567 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=edcc0b5dc968f7ed1ca7e4cba79b64fb, ASSIGN 2023-07-29 02:59:22,569 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=edcc0b5dc968f7ed1ca7e4cba79b64fb, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,41957,1690599561607; forceNewPlan=false, retain=false 2023-07-29 02:59:22,720 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=edcc0b5dc968f7ed1ca7e4cba79b64fb, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,721 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599562720"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599562720"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599562720"}]},"ts":"1690599562720"} 2023-07-29 02:59:22,726 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure edcc0b5dc968f7ed1ca7e4cba79b64fb, server=jenkins-hbase16.apache.org,41957,1690599561607}] 2023-07-29 02:59:22,892 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:22,892 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => edcc0b5dc968f7ed1ca7e4cba79b64fb, NAME => 'hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb.', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:59:22,893 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,893 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:22,893 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,893 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,896 INFO [StoreOpener-edcc0b5dc968f7ed1ca7e4cba79b64fb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,898 DEBUG [StoreOpener-edcc0b5dc968f7ed1ca7e4cba79b64fb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/info 2023-07-29 02:59:22,898 DEBUG [StoreOpener-edcc0b5dc968f7ed1ca7e4cba79b64fb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/info 2023-07-29 02:59:22,899 INFO [StoreOpener-edcc0b5dc968f7ed1ca7e4cba79b64fb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region edcc0b5dc968f7ed1ca7e4cba79b64fb columnFamilyName info 2023-07-29 02:59:22,899 INFO [StoreOpener-edcc0b5dc968f7ed1ca7e4cba79b64fb-1] regionserver.HStore(310): Store=edcc0b5dc968f7ed1ca7e4cba79b64fb/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:22,900 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,901 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,905 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 02:59:22,908 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:59:22,909 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened edcc0b5dc968f7ed1ca7e4cba79b64fb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=700986, jitterRate=-0.1086508184671402}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 02:59:22,909 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for edcc0b5dc968f7ed1ca7e4cba79b64fb: 2023-07-29 02:59:22,910 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb., pid=6, masterSystemTime=1690599562882 2023-07-29 02:59:22,914 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:22,914 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:22,917 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=edcc0b5dc968f7ed1ca7e4cba79b64fb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:22,917 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599562917"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599562917"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599562917"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599562917"}]},"ts":"1690599562917"} 2023-07-29 02:59:22,925 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-29 02:59:22,925 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure edcc0b5dc968f7ed1ca7e4cba79b64fb, server=jenkins-hbase16.apache.org,41957,1690599561607 in 195 msec 2023-07-29 02:59:22,929 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-29 02:59:22,929 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=edcc0b5dc968f7ed1ca7e4cba79b64fb, ASSIGN in 362 msec 2023-07-29 02:59:22,937 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-29 02:59:22,938 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599562937"}]},"ts":"1690599562937"} 2023-07-29 02:59:22,940 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-29 02:59:22,947 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-29 02:59:22,949 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 490 msec 2023-07-29 02:59:22,961 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-29 02:59:23,013 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-29 02:59:23,013 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:23,020 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-29 02:59:23,037 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 02:59:23,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 29 msec 2023-07-29 02:59:23,064 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-29 02:59:23,079 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 02:59:23,094 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 28 msec 2023-07-29 02:59:23,112 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-29 02:59:23,129 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-29 02:59:23,129 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.466sec 2023-07-29 02:59:23,129 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-29 02:59:23,130 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-29 02:59:23,130 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-29 02:59:23,130 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41613,1690599561459-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-29 02:59:23,130 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41613,1690599561459-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-29 02:59:23,134 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-29 02:59:23,142 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ReadOnlyZKClient(139): Connect 0x445b3ad8 to 127.0.0.1:49643 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 02:59:23,155 DEBUG [Listener at localhost.localdomain/41871] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@107a3a09, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 02:59:23,157 DEBUG [hconnection-0xae9169c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 02:59:23,159 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:54816, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 02:59:23,162 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 02:59:23,163 INFO [Listener at localhost.localdomain/41871] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 02:59:23,179 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-29 02:59:23,179 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 02:59:23,180 INFO [Listener at localhost.localdomain/41871] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-07-29 02:59:23,183 DEBUG [Listener at localhost.localdomain/41871] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-29 02:59:23,187 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:48856, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-29 02:59:23,190 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-07-29 02:59:23,190 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-07-29 02:59:23,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-29 02:59:23,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:23,195 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-07-29 02:59:23,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-07-29 02:59:23,196 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-29 02:59:23,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-29 02:59:23,199 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,200 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b empty. 2023-07-29 02:59:23,200 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,201 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-07-29 02:59:23,223 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-07-29 02:59:23,225 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 280585b1f353e908144dd0473109203b, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/.tmp 2023-07-29 02:59:23,247 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:23,248 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing 280585b1f353e908144dd0473109203b, disabling compactions & flushes 2023-07-29 02:59:23,248 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:23,248 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:23,248 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. after waiting 0 ms 2023-07-29 02:59:23,248 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:23,248 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:23,248 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for 280585b1f353e908144dd0473109203b: 2023-07-29 02:59:23,251 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-07-29 02:59:23,253 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1690599563253"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599563253"}]},"ts":"1690599563253"} 2023-07-29 02:59:23,255 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-29 02:59:23,257 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-29 02:59:23,257 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599563257"}]},"ts":"1690599563257"} 2023-07-29 02:59:23,260 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-07-29 02:59:23,280 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=280585b1f353e908144dd0473109203b, ASSIGN}] 2023-07-29 02:59:23,284 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=280585b1f353e908144dd0473109203b, ASSIGN 2023-07-29 02:59:23,288 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=280585b1f353e908144dd0473109203b, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,41957,1690599561607; forceNewPlan=false, retain=false 2023-07-29 02:59:23,439 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=280585b1f353e908144dd0473109203b, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:23,439 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1690599563439"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599563439"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599563439"}]},"ts":"1690599563439"} 2023-07-29 02:59:23,442 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 280585b1f353e908144dd0473109203b, server=jenkins-hbase16.apache.org,41957,1690599561607}] 2023-07-29 02:59:23,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:23,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 280585b1f353e908144dd0473109203b, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.', STARTKEY => '', ENDKEY => ''} 2023-07-29 02:59:23,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling 280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 02:59:23,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,606 INFO [StoreOpener-280585b1f353e908144dd0473109203b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,610 DEBUG [StoreOpener-280585b1f353e908144dd0473109203b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info 2023-07-29 02:59:23,610 DEBUG [StoreOpener-280585b1f353e908144dd0473109203b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info 2023-07-29 02:59:23,610 INFO [StoreOpener-280585b1f353e908144dd0473109203b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 280585b1f353e908144dd0473109203b columnFamilyName info 2023-07-29 02:59:23,611 INFO [StoreOpener-280585b1f353e908144dd0473109203b-1] regionserver.HStore(310): Store=280585b1f353e908144dd0473109203b/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 02:59:23,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 280585b1f353e908144dd0473109203b 2023-07-29 02:59:23,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 02:59:23,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 280585b1f353e908144dd0473109203b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=755769, jitterRate=-0.038990288972854614}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 02:59:23,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 280585b1f353e908144dd0473109203b: 2023-07-29 02:59:23,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b., pid=11, masterSystemTime=1690599563595 2023-07-29 02:59:23,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:23,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:23,625 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=280585b1f353e908144dd0473109203b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:23,625 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1690599563625"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599563625"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599563625"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599563625"}]},"ts":"1690599563625"} 2023-07-29 02:59:23,630 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-29 02:59:23,630 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 280585b1f353e908144dd0473109203b, server=jenkins-hbase16.apache.org,41957,1690599561607 in 186 msec 2023-07-29 02:59:23,633 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-29 02:59:23,633 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=280585b1f353e908144dd0473109203b, ASSIGN in 350 msec 2023-07-29 02:59:23,634 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-29 02:59:23,634 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599563634"}]},"ts":"1690599563634"} 2023-07-29 02:59:23,636 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-07-29 02:59:23,647 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-07-29 02:59:23,649 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 457 msec 2023-07-29 02:59:25,190 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 02:59:25,190 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-29 02:59:27,903 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-29 02:59:28,021 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-29 02:59:28,021 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-29 02:59:28,022 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 02:59:33,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-29 02:59:33,202 INFO [Listener at localhost.localdomain/41871] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-07-29 02:59:33,209 DEBUG [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:33,209 DEBUG [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:33,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-07-29 02:59:33,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-07-29 02:59:33,228 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-07-29 02:59:33,228 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 02:59:33,229 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-07-29 02:59:33,229 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-07-29 02:59:33,230 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,230 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-07-29 02:59:33,262 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 02:59:33,262 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 02:59:33,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:59:33,263 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,263 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-29 02:59:33,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-07-29 02:59:33,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-29 02:59:33,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-07-29 02:59:33,266 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-07-29 02:59:33,269 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-07-29 02:59:33,269 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-07-29 02:59:33,269 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 02:59:33,270 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-07-29 02:59:33,271 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-29 02:59:33,271 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-29 02:59:33,271 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:33,272 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. started... 2023-07-29 02:59:33,272 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing edcc0b5dc968f7ed1ca7e4cba79b64fb 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-29 02:59:33,289 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/.tmp/info/fad6ee92af0d401eb3c1b07a377c0f2b 2023-07-29 02:59:33,300 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/.tmp/info/fad6ee92af0d401eb3c1b07a377c0f2b as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/info/fad6ee92af0d401eb3c1b07a377c0f2b 2023-07-29 02:59:33,311 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/info/fad6ee92af0d401eb3c1b07a377c0f2b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-29 02:59:33,312 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for edcc0b5dc968f7ed1ca7e4cba79b64fb in 40ms, sequenceid=6, compaction requested=false 2023-07-29 02:59:33,313 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for edcc0b5dc968f7ed1ca7e4cba79b64fb: 2023-07-29 02:59:33,313 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 02:59:33,313 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-29 02:59:33,313 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-29 02:59:33,313 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,314 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-07-29 02:59:33,314 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure (hbase:namespace) in zk 2023-07-29 02:59:33,320 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,320 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:33,321 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,321 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-29 02:59:33,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:33,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:33,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-07-29 02:59:33,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:33,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-07-29 02:59:33,323 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-07-29 02:59:33,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5800b6e4[Count = 0] remaining members to acquire global barrier 2023-07-29 02:59:33,323 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,334 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,334 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-07-29 02:59:33,335 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-07-29 02:59:33,335 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,335 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-29 02:59:33,335 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase16.apache.org,41957,1690599561607' in zk 2023-07-29 02:59:33,345 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-07-29 02:59:33,345 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,345 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 02:59:33,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:33,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:33,346 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-07-29 02:59:33,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:33,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:33,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-07-29 02:59:33,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:33,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-07-29 02:59:33,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase16.apache.org,41957,1690599561607': 2023-07-29 02:59:33,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-07-29 02:59:33,350 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-07-29 02:59:33,350 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-29 02:59:33,350 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-29 02:59:33,350 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-07-29 02:59:33,350 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-29 02:59:33,359 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,359 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:33,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:33,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,359 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 02:59:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:33,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 02:59:33,360 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:59:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-07-29 02:59:33,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:33,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-07-29 02:59:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,362 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:33,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-07-29 02:59:33,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,376 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,376 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 02:59:33,376 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-07-29 02:59:33,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 02:59:33,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:59:33,376 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 02:59:33,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-07-29 02:59:33,376 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 02:59:33,376 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-07-29 02:59:33,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-29 02:59:33,377 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:33,377 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 02:59:33,377 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-07-29 02:59:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:59:33,377 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-07-29 02:59:33,378 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-07-29 02:59:33,379 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-07-29 02:59:43,379 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-29 02:59:43,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-29 02:59:43,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-07-29 02:59:43,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,404 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 02:59:43,404 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 02:59:43,404 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-07-29 02:59:43,405 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-07-29 02:59:43,405 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,405 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,545 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 02:59:43,545 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 02:59:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:59:43,546 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,546 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-29 02:59:43,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-29 02:59:43,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,549 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-07-29 02:59:43,549 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 02:59:43,550 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-07-29 02:59:43,551 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-29 02:59:43,551 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-29 02:59:43,551 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:43,551 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. started... 2023-07-29 02:59:43,552 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 280585b1f353e908144dd0473109203b 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-07-29 02:59:43,569 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/a2828b41591947428495add23f225baf 2023-07-29 02:59:43,579 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/a2828b41591947428495add23f225baf as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a2828b41591947428495add23f225baf 2023-07-29 02:59:43,586 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a2828b41591947428495add23f225baf, entries=1, sequenceid=5, filesize=5.8 K 2023-07-29 02:59:43,588 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 280585b1f353e908144dd0473109203b in 36ms, sequenceid=5, compaction requested=false 2023-07-29 02:59:43,588 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 280585b1f353e908144dd0473109203b: 2023-07-29 02:59:43,589 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:43,589 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-29 02:59:43,589 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-29 02:59:43,589 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,589 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-07-29 02:59:43,589 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-07-29 02:59:43,595 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,595 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:43,595 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,595 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-29 02:59:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:43,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-07-29 02:59:43,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c3f393c[Count = 0] remaining members to acquire global barrier 2023-07-29 02:59:43,598 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-07-29 02:59:43,598 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,603 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,603 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-07-29 02:59:43,603 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-07-29 02:59:43,603 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase16.apache.org,41957,1690599561607' in zk 2023-07-29 02:59:43,604 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,604 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-29 02:59:43,611 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-07-29 02:59:43,612 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 02:59:43,612 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-07-29 02:59:43,612 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase16.apache.org,41957,1690599561607': 2023-07-29 02:59:43,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-07-29 02:59:43,615 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-07-29 02:59:43,615 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-29 02:59:43,615 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-29 02:59:43,615 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,615 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-29 02:59:43,640 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,640 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:43,640 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 02:59:43,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 02:59:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:59:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:43,641 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,643 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,653 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,653 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 02:59:43,653 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 02:59:43,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:59:43,653 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 02:59:43,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 02:59:43,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-29 02:59:43,653 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 02:59:43,653 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 02:59:43,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:59:43,654 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:43,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-07-29 02:59:43,655 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-07-29 02:59:43,655 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-07-29 02:59:43,655 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,655 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:43,655 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,655 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-29 02:59:53,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-29 02:59:53,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-07-29 02:59:53,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-07-29 02:59:53,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,670 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 02:59:53,670 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 02:59:53,670 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-07-29 02:59:53,670 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-07-29 02:59:53,671 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,671 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,703 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 02:59:53,703 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 02:59:53,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:59:53,704 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,704 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-29 02:59:53,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,704 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-29 02:59:53,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,705 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-07-29 02:59:53,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,706 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-07-29 02:59:53,706 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 02:59:53,707 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-07-29 02:59:53,707 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-29 02:59:53,707 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-29 02:59:53,707 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:53,708 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. started... 2023-07-29 02:59:53,708 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 280585b1f353e908144dd0473109203b 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-07-29 02:59:53,726 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/74328804fee04b1cbd6e422eee2e8607 2023-07-29 02:59:53,734 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/74328804fee04b1cbd6e422eee2e8607 as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/74328804fee04b1cbd6e422eee2e8607 2023-07-29 02:59:53,741 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/74328804fee04b1cbd6e422eee2e8607, entries=1, sequenceid=9, filesize=5.8 K 2023-07-29 02:59:53,742 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 280585b1f353e908144dd0473109203b in 34ms, sequenceid=9, compaction requested=false 2023-07-29 02:59:53,742 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 280585b1f353e908144dd0473109203b: 2023-07-29 02:59:53,742 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 02:59:53,742 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-29 02:59:53,743 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-29 02:59:53,743 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,743 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-07-29 02:59:53,743 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-07-29 02:59:53,753 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,753 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:53,753 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,753 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-29 02:59:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:53,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-07-29 02:59:53,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@193dbcdf[Count = 0] remaining members to acquire global barrier 2023-07-29 02:59:53,756 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-07-29 02:59:53,756 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,761 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,761 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-07-29 02:59:53,762 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-07-29 02:59:53,762 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase16.apache.org,41957,1690599561607' in zk 2023-07-29 02:59:53,762 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,762 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-29 02:59:53,769 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-07-29 02:59:53,769 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 02:59:53,769 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-07-29 02:59:53,769 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:53,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:53,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:53,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:53,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase16.apache.org,41957,1690599561607': 2023-07-29 02:59:53,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-07-29 02:59:53,773 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-07-29 02:59:53,773 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-29 02:59:53,773 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-29 02:59:53,773 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,773 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-29 02:59:53,783 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,783 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 02:59:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 02:59:53,784 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 02:59:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 02:59:53,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 02:59:53,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:59:53,785 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 02:59:53,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,787 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 02:59:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,795 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,795 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 02:59:53,795 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 02:59:53,795 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 02:59:53,795 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 02:59:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 02:59:53,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 02:59:53,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-29 02:59:53,795 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,796 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 02:59:53,796 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,796 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 02:59:53,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 02:59:53,796 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 02:59:53,797 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-07-29 02:59:53,797 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-07-29 03:00:03,797 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-29 03:00:03,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-29 03:00:03,820 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK] 2023-07-29 03:00:03,820 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK] 2023-07-29 03:00:03,823 INFO [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599562158 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599603801 2023-07-29 03:00:03,824 DEBUG [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK], DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK]] 2023-07-29 03:00:03,824 DEBUG [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599562158 is not closed yet, will try archiving it next time 2023-07-29 03:00:03,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-07-29 03:00:03,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-07-29 03:00:03,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,832 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 03:00:03,832 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 03:00:03,832 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-07-29 03:00:03,832 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-07-29 03:00:03,833 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,833 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,869 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 03:00:03,869 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 03:00:03,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 03:00:03,870 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,870 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-29 03:00:03,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-29 03:00:03,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-07-29 03:00:03,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,871 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-07-29 03:00:03,871 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 03:00:03,871 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-07-29 03:00:03,872 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-29 03:00:03,872 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-29 03:00:03,872 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:03,872 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. started... 2023-07-29 03:00:03,872 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 280585b1f353e908144dd0473109203b 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-07-29 03:00:03,883 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/53c15a3629644e53a898df9b7902b169 2023-07-29 03:00:03,891 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/53c15a3629644e53a898df9b7902b169 as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/53c15a3629644e53a898df9b7902b169 2023-07-29 03:00:03,898 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/53c15a3629644e53a898df9b7902b169, entries=1, sequenceid=13, filesize=5.8 K 2023-07-29 03:00:03,899 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 280585b1f353e908144dd0473109203b in 27ms, sequenceid=13, compaction requested=true 2023-07-29 03:00:03,899 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 280585b1f353e908144dd0473109203b: 2023-07-29 03:00:03,899 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:03,899 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-29 03:00:03,900 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-29 03:00:03,900 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,900 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-07-29 03:00:03,900 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-07-29 03:00:03,908 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,908 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 03:00:03,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 03:00:03,908 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,908 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-29 03:00:03,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 03:00:03,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 03:00:03,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 03:00:03,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-07-29 03:00:03,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7584d886[Count = 0] remaining members to acquire global barrier 2023-07-29 03:00:03,910 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-07-29 03:00:03,910 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,919 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,919 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-07-29 03:00:03,919 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-07-29 03:00:03,919 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase16.apache.org,41957,1690599561607' in zk 2023-07-29 03:00:03,919 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,919 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-29 03:00:03,927 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-07-29 03:00:03,927 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,927 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 03:00:03,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 03:00:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 03:00:03,927 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-07-29 03:00:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 03:00:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 03:00:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 03:00:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase16.apache.org,41957,1690599561607': 2023-07-29 03:00:03,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-07-29 03:00:03,930 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-07-29 03:00:03,931 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-29 03:00:03,931 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-29 03:00:03,931 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,931 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-29 03:00:03,941 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,941 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 03:00:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 03:00:03,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,941 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 03:00:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 03:00:03,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 03:00:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 03:00:03,942 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 03:00:03,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,943 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 03:00:03,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,952 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,952 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 03:00:03,952 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,952 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 03:00:03,952 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,952 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 03:00:03,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 03:00:03,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 03:00:03,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 03:00:03,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-29 03:00:03,952 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:03,953 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,953 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,953 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:03,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 03:00:03,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 03:00:03,954 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-07-29 03:00:03,954 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-07-29 03:00:13,954 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-29 03:00:13,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-29 03:00:13,957 DEBUG [Listener at localhost.localdomain/41871] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:00:13,972 DEBUG [Listener at localhost.localdomain/41871] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:00:13,975 DEBUG [Listener at localhost.localdomain/41871] regionserver.HStore(1912): 280585b1f353e908144dd0473109203b/info is initiating minor compaction (all files) 2023-07-29 03:00:13,975 INFO [Listener at localhost.localdomain/41871] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-29 03:00:13,975 INFO [Listener at localhost.localdomain/41871] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:13,978 INFO [Listener at localhost.localdomain/41871] regionserver.HRegion(2259): Starting compaction of 280585b1f353e908144dd0473109203b/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:13,979 INFO [Listener at localhost.localdomain/41871] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a2828b41591947428495add23f225baf, hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/74328804fee04b1cbd6e422eee2e8607, hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/53c15a3629644e53a898df9b7902b169] into tmpdir=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp, totalSize=17.3 K 2023-07-29 03:00:13,981 DEBUG [Listener at localhost.localdomain/41871] compactions.Compactor(207): Compacting a2828b41591947428495add23f225baf, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1690599583395 2023-07-29 03:00:13,982 DEBUG [Listener at localhost.localdomain/41871] compactions.Compactor(207): Compacting 74328804fee04b1cbd6e422eee2e8607, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1690599593660 2023-07-29 03:00:13,982 DEBUG [Listener at localhost.localdomain/41871] compactions.Compactor(207): Compacting 53c15a3629644e53a898df9b7902b169, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1690599603799 2023-07-29 03:00:14,024 INFO [Listener at localhost.localdomain/41871] throttle.PressureAwareThroughputController(145): 280585b1f353e908144dd0473109203b#info#compaction#9 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:00:14,081 DEBUG [Listener at localhost.localdomain/41871] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/b52d1209462540079defe0eb0df98341 as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/b52d1209462540079defe0eb0df98341 2023-07-29 03:00:14,113 INFO [Listener at localhost.localdomain/41871] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 280585b1f353e908144dd0473109203b/info of 280585b1f353e908144dd0473109203b into b52d1209462540079defe0eb0df98341(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:00:14,114 DEBUG [Listener at localhost.localdomain/41871] regionserver.HRegion(2289): Compaction status journal for 280585b1f353e908144dd0473109203b: 2023-07-29 03:00:14,145 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK] 2023-07-29 03:00:14,154 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK] 2023-07-29 03:00:14,163 INFO [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599603801 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599614116 2023-07-29 03:00:14,164 DEBUG [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK], DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK]] 2023-07-29 03:00:14,164 DEBUG [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599603801 is not closed yet, will try archiving it next time 2023-07-29 03:00:14,164 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599562158 to hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/oldWALs/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599562158 2023-07-29 03:00:14,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-07-29 03:00:14,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-07-29 03:00:14,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,185 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 03:00:14,185 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 03:00:14,186 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-07-29 03:00:14,186 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-07-29 03:00:14,187 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,187 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,242 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 03:00:14,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 03:00:14,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 03:00:14,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,244 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,245 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,245 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-29 03:00:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-29 03:00:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,246 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-07-29 03:00:14,246 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,247 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-07-29 03:00:14,247 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-29 03:00:14,247 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-07-29 03:00:14,249 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-29 03:00:14,249 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-29 03:00:14,249 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:14,249 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. started... 2023-07-29 03:00:14,249 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 280585b1f353e908144dd0473109203b 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-07-29 03:00:14,280 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/292ed450df8243d69e912220eb25c5aa 2023-07-29 03:00:14,303 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/292ed450df8243d69e912220eb25c5aa as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/292ed450df8243d69e912220eb25c5aa 2023-07-29 03:00:14,323 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/292ed450df8243d69e912220eb25c5aa, entries=1, sequenceid=18, filesize=5.8 K 2023-07-29 03:00:14,325 INFO [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 280585b1f353e908144dd0473109203b in 75ms, sequenceid=18, compaction requested=false 2023-07-29 03:00:14,326 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 280585b1f353e908144dd0473109203b: 2023-07-29 03:00:14,326 DEBUG [rs(jenkins-hbase16.apache.org,41957,1690599561607)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:14,326 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-29 03:00:14,329 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-29 03:00:14,329 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,329 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-07-29 03:00:14,329 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-07-29 03:00:14,344 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 03:00:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 03:00:14,345 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 03:00:14,345 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,345 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-29 03:00:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 03:00:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 03:00:14,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,41957,1690599561607' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-07-29 03:00:14,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@286ee744[Count = 0] remaining members to acquire global barrier 2023-07-29 03:00:14,347 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-07-29 03:00:14,347 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,358 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,358 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-07-29 03:00:14,358 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-07-29 03:00:14,358 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase16.apache.org,41957,1690599561607' in zk 2023-07-29 03:00:14,358 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,358 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-29 03:00:14,369 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-07-29 03:00:14,369 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 03:00:14,369 DEBUG [member: 'jenkins-hbase16.apache.org,41957,1690599561607' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-07-29 03:00:14,370 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 03:00:14,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 03:00:14,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 03:00:14,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 03:00:14,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 03:00:14,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase16.apache.org,41957,1690599561607': 2023-07-29 03:00:14,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,41957,1690599561607' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-07-29 03:00:14,375 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-07-29 03:00:14,375 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-29 03:00:14,375 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-29 03:00:14,375 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,375 INFO [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-29 03:00:14,385 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,386 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-29 03:00:14,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-29 03:00:14,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,386 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 03:00:14,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-29 03:00:14,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 03:00:14,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 03:00:14,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,387 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-29 03:00:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-29 03:00:14,392 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,408 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-29 03:00:14,408 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-29 03:00:14,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-29 03:00:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 03:00:14,410 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,410 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,410 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,410 DEBUG [(jenkins-hbase16.apache.org,41613,1690599561459)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-29 03:00:14,410 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:14,410 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,410 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,410 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-07-29 03:00:14,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-07-29 03:00:14,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-29 03:00:14,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-29 03:00:14,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 03:00:14,412 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-07-29 03:00:14,412 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-07-29 03:00:24,413 DEBUG [Listener at localhost.localdomain/41871] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-29 03:00:24,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41613] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-29 03:00:24,441 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK] 2023-07-29 03:00:24,441 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK] 2023-07-29 03:00:24,443 INFO [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599614116 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599624423 2023-07-29 03:00:24,443 DEBUG [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46401,DS-863478f6-b727-42d4-a199-c252c0df1f8d,DISK], DatanodeInfoWithStorage[127.0.0.1:37773,DS-97cf72b3-554a-4b26-ac2d-6b48bf182471,DISK]] 2023-07-29 03:00:24,443 DEBUG [Listener at localhost.localdomain/41871] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599614116 is not closed yet, will try archiving it next time 2023-07-29 03:00:24,444 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-29 03:00:24,444 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/WALs/jenkins-hbase16.apache.org,41957,1690599561607/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599603801 to hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/oldWALs/jenkins-hbase16.apache.org%2C41957%2C1690599561607.1690599603801 2023-07-29 03:00:24,444 INFO [Listener at localhost.localdomain/41871] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-29 03:00:24,444 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x445b3ad8 to 127.0.0.1:49643 2023-07-29 03:00:24,445 DEBUG [Listener at localhost.localdomain/41871] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:00:24,445 DEBUG [Listener at localhost.localdomain/41871] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-29 03:00:24,445 DEBUG [Listener at localhost.localdomain/41871] util.JVMClusterUtil(257): Found active master hash=1696319861, stopped=false 2023-07-29 03:00:24,445 INFO [Listener at localhost.localdomain/41871] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 03:00:24,482 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 03:00:24,482 INFO [Listener at localhost.localdomain/41871] procedure2.ProcedureExecutor(629): Stopping 2023-07-29 03:00:24,482 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 03:00:24,482 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:24,483 DEBUG [Listener at localhost.localdomain/41871] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f3eafee to 127.0.0.1:49643 2023-07-29 03:00:24,483 DEBUG [Listener at localhost.localdomain/41871] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:00:24,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:00:24,483 INFO [Listener at localhost.localdomain/41871] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,41957,1690599561607' ***** 2023-07-29 03:00:24,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:00:24,484 INFO [Listener at localhost.localdomain/41871] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-29 03:00:24,484 INFO [RS:0;jenkins-hbase16:41957] regionserver.HeapMemoryManager(220): Stopping 2023-07-29 03:00:24,484 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-29 03:00:24,484 INFO [RS:0;jenkins-hbase16:41957] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-29 03:00:24,484 INFO [RS:0;jenkins-hbase16:41957] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-29 03:00:24,484 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(3305): Received CLOSE for edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 03:00:24,485 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(3305): Received CLOSE for 280585b1f353e908144dd0473109203b 2023-07-29 03:00:24,485 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:24,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing edcc0b5dc968f7ed1ca7e4cba79b64fb, disabling compactions & flushes 2023-07-29 03:00:24,485 DEBUG [RS:0;jenkins-hbase16:41957] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x233103ae to 127.0.0.1:49643 2023-07-29 03:00:24,485 DEBUG [RS:0;jenkins-hbase16:41957] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:00:24,485 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 03:00:24,485 INFO [RS:0;jenkins-hbase16:41957] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-29 03:00:24,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 03:00:24,485 INFO [RS:0;jenkins-hbase16:41957] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-29 03:00:24,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. after waiting 0 ms 2023-07-29 03:00:24,485 INFO [RS:0;jenkins-hbase16:41957] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-29 03:00:24,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 03:00:24,485 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-29 03:00:24,485 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-29 03:00:24,485 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, edcc0b5dc968f7ed1ca7e4cba79b64fb=hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb., 280585b1f353e908144dd0473109203b=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.} 2023-07-29 03:00:24,486 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 03:00:24,487 DEBUG [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1504): Waiting on 1588230740, 280585b1f353e908144dd0473109203b, edcc0b5dc968f7ed1ca7e4cba79b64fb 2023-07-29 03:00:24,487 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 03:00:24,487 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 03:00:24,487 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 03:00:24,487 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 03:00:24,487 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-07-29 03:00:24,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/namespace/edcc0b5dc968f7ed1ca7e4cba79b64fb/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-29 03:00:24,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 03:00:24,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for edcc0b5dc968f7ed1ca7e4cba79b64fb: 2023-07-29 03:00:24,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690599562456.edcc0b5dc968f7ed1ca7e4cba79b64fb. 2023-07-29 03:00:24,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 280585b1f353e908144dd0473109203b, disabling compactions & flushes 2023-07-29 03:00:24,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:24,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:24,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. after waiting 0 ms 2023-07-29 03:00:24,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:24,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 280585b1f353e908144dd0473109203b 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-07-29 03:00:24,506 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.85 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/.tmp/info/da3bbc25b99a410191000d34b859ba86 2023-07-29 03:00:24,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/a3c7658188994febad5f30b208d8e703 2023-07-29 03:00:24,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/.tmp/info/a3c7658188994febad5f30b208d8e703 as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a3c7658188994febad5f30b208d8e703 2023-07-29 03:00:24,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a3c7658188994febad5f30b208d8e703, entries=1, sequenceid=22, filesize=5.8 K 2023-07-29 03:00:24,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 280585b1f353e908144dd0473109203b in 35ms, sequenceid=22, compaction requested=true 2023-07-29 03:00:24,532 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/.tmp/table/48abb58c257d4601a5cd8f63940ea600 2023-07-29 03:00:24,533 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a2828b41591947428495add23f225baf, hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/74328804fee04b1cbd6e422eee2e8607, hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/53c15a3629644e53a898df9b7902b169] to archive 2023-07-29 03:00:24,534 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-29 03:00:24,537 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a2828b41591947428495add23f225baf to hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/a2828b41591947428495add23f225baf 2023-07-29 03:00:24,538 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/74328804fee04b1cbd6e422eee2e8607 to hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/74328804fee04b1cbd6e422eee2e8607 2023-07-29 03:00:24,540 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/53c15a3629644e53a898df9b7902b169 to hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/info/53c15a3629644e53a898df9b7902b169 2023-07-29 03:00:24,541 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/.tmp/info/da3bbc25b99a410191000d34b859ba86 as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/info/da3bbc25b99a410191000d34b859ba86 2023-07-29 03:00:24,549 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/info/da3bbc25b99a410191000d34b859ba86, entries=20, sequenceid=14, filesize=7.6 K 2023-07-29 03:00:24,551 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/.tmp/table/48abb58c257d4601a5cd8f63940ea600 as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/table/48abb58c257d4601a5cd8f63940ea600 2023-07-29 03:00:24,558 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/table/48abb58c257d4601a5cd8f63940ea600, entries=4, sequenceid=14, filesize=4.9 K 2023-07-29 03:00:24,559 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3178, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 72ms, sequenceid=14, compaction requested=false 2023-07-29 03:00:24,567 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-07-29 03:00:24,567 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 03:00:24,568 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 03:00:24,568 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 03:00:24,568 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-29 03:00:24,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/280585b1f353e908144dd0473109203b/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-07-29 03:00:24,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:24,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 280585b1f353e908144dd0473109203b: 2023-07-29 03:00:24,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1690599563190.280585b1f353e908144dd0473109203b. 2023-07-29 03:00:24,687 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,41957,1690599561607; all regions closed. 2023-07-29 03:00:24,696 DEBUG [RS:0;jenkins-hbase16:41957] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/oldWALs 2023-07-29 03:00:24,697 INFO [RS:0;jenkins-hbase16:41957] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C41957%2C1690599561607.meta:.meta(num 1690599562349) 2023-07-29 03:00:24,705 DEBUG [RS:0;jenkins-hbase16:41957] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/oldWALs 2023-07-29 03:00:24,705 INFO [RS:0;jenkins-hbase16:41957] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C41957%2C1690599561607:(num 1690599624423) 2023-07-29 03:00:24,705 DEBUG [RS:0;jenkins-hbase16:41957] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:00:24,705 INFO [RS:0;jenkins-hbase16:41957] regionserver.LeaseManager(133): Closed leases 2023-07-29 03:00:24,706 INFO [RS:0;jenkins-hbase16:41957] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-29 03:00:24,706 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 03:00:24,706 INFO [RS:0;jenkins-hbase16:41957] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:41957 2023-07-29 03:00:24,715 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,41957,1690599561607 2023-07-29 03:00:24,715 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:00:24,715 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:00:24,716 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,41957,1690599561607] 2023-07-29 03:00:24,716 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,41957,1690599561607; numProcessing=1 2023-07-29 03:00:24,732 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,41957,1690599561607 already deleted, retry=false 2023-07-29 03:00:24,732 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,41957,1690599561607 expired; onlineServers=0 2023-07-29 03:00:24,732 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,41613,1690599561459' ***** 2023-07-29 03:00:24,732 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-29 03:00:24,733 DEBUG [M:0;jenkins-hbase16:41613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6124510e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 03:00:24,733 INFO [M:0;jenkins-hbase16:41613] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,41613,1690599561459 2023-07-29 03:00:24,733 INFO [M:0;jenkins-hbase16:41613] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,41613,1690599561459; all regions closed. 2023-07-29 03:00:24,733 DEBUG [M:0;jenkins-hbase16:41613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:00:24,733 DEBUG [M:0;jenkins-hbase16:41613] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-29 03:00:24,733 DEBUG [M:0;jenkins-hbase16:41613] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-29 03:00:24,733 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-29 03:00:24,733 INFO [M:0;jenkins-hbase16:41613] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-29 03:00:24,734 INFO [M:0;jenkins-hbase16:41613] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-29 03:00:24,733 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599561934] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599561934,5,FailOnTimeoutGroup] 2023-07-29 03:00:24,734 INFO [M:0;jenkins-hbase16:41613] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-07-29 03:00:24,733 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599561934] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599561934,5,FailOnTimeoutGroup] 2023-07-29 03:00:24,734 DEBUG [M:0;jenkins-hbase16:41613] master.HMaster(1512): Stopping service threads 2023-07-29 03:00:24,735 INFO [M:0;jenkins-hbase16:41613] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-29 03:00:24,735 ERROR [M:0;jenkins-hbase16:41613] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-29 03:00:24,736 INFO [M:0;jenkins-hbase16:41613] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-29 03:00:24,736 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-29 03:00:24,743 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-29 03:00:24,743 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:24,743 DEBUG [M:0;jenkins-hbase16:41613] zookeeper.ZKUtil(398): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-29 03:00:24,744 WARN [M:0;jenkins-hbase16:41613] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-29 03:00:24,744 INFO [M:0;jenkins-hbase16:41613] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-29 03:00:24,744 INFO [M:0;jenkins-hbase16:41613] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-29 03:00:24,745 DEBUG [M:0;jenkins-hbase16:41613] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 03:00:24,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 03:00:24,745 INFO [M:0;jenkins-hbase16:41613] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:24,745 DEBUG [M:0;jenkins-hbase16:41613] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:24,745 DEBUG [M:0;jenkins-hbase16:41613] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 03:00:24,745 DEBUG [M:0;jenkins-hbase16:41613] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:24,746 INFO [M:0;jenkins-hbase16:41613] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.94 KB heapSize=47.38 KB 2023-07-29 03:00:24,767 INFO [M:0;jenkins-hbase16:41613] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.94 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b5d9b6116d264d70bec5f043978419be 2023-07-29 03:00:24,773 INFO [M:0;jenkins-hbase16:41613] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5d9b6116d264d70bec5f043978419be 2023-07-29 03:00:24,775 DEBUG [M:0;jenkins-hbase16:41613] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b5d9b6116d264d70bec5f043978419be as hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b5d9b6116d264d70bec5f043978419be 2023-07-29 03:00:24,780 INFO [M:0;jenkins-hbase16:41613] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5d9b6116d264d70bec5f043978419be 2023-07-29 03:00:24,781 INFO [M:0;jenkins-hbase16:41613] regionserver.HStore(1080): Added hdfs://localhost.localdomain:40955/user/jenkins/test-data/2e71ac77-d421-beed-72a6-24dd4af07772/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b5d9b6116d264d70bec5f043978419be, entries=11, sequenceid=100, filesize=6.1 K 2023-07-29 03:00:24,782 INFO [M:0;jenkins-hbase16:41613] regionserver.HRegion(2948): Finished flush of dataSize ~38.94 KB/39878, heapSize ~47.36 KB/48496, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 37ms, sequenceid=100, compaction requested=false 2023-07-29 03:00:24,783 INFO [M:0;jenkins-hbase16:41613] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:24,783 DEBUG [M:0;jenkins-hbase16:41613] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 03:00:24,786 INFO [M:0;jenkins-hbase16:41613] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-29 03:00:24,786 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 03:00:24,786 INFO [M:0;jenkins-hbase16:41613] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:41613 2023-07-29 03:00:24,793 DEBUG [M:0;jenkins-hbase16:41613] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,41613,1690599561459 already deleted, retry=false 2023-07-29 03:00:24,824 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:00:24,824 INFO [RS:0;jenkins-hbase16:41957] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,41957,1690599561607; zookeeper connection closed. 2023-07-29 03:00:24,824 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): regionserver:41957-0x101af0457210001, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:00:24,825 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7d36b83] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7d36b83 2023-07-29 03:00:24,825 INFO [Listener at localhost.localdomain/41871] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-29 03:00:24,924 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:00:24,924 INFO [M:0;jenkins-hbase16:41613] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,41613,1690599561459; zookeeper connection closed. 2023-07-29 03:00:24,925 DEBUG [Listener at localhost.localdomain/41871-EventThread] zookeeper.ZKWatcher(600): master:41613-0x101af0457210000, quorum=127.0.0.1:49643, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:00:24,927 WARN [Listener at localhost.localdomain/41871] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 03:00:24,957 INFO [Listener at localhost.localdomain/41871] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 03:00:25,067 WARN [BP-885770418-188.40.62.62-1690599559790 heartbeating to localhost.localdomain/127.0.0.1:40955] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 03:00:25,067 WARN [BP-885770418-188.40.62.62-1690599559790 heartbeating to localhost.localdomain/127.0.0.1:40955] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-885770418-188.40.62.62-1690599559790 (Datanode Uuid 36071cb8-68c0-4991-8095-3d127e30f924) service to localhost.localdomain/127.0.0.1:40955 2023-07-29 03:00:25,069 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686/dfs/data/data3/current/BP-885770418-188.40.62.62-1690599559790] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:00:25,070 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686/dfs/data/data4/current/BP-885770418-188.40.62.62-1690599559790] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:00:25,072 WARN [Listener at localhost.localdomain/41871] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 03:00:25,075 INFO [Listener at localhost.localdomain/41871] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 03:00:25,183 WARN [BP-885770418-188.40.62.62-1690599559790 heartbeating to localhost.localdomain/127.0.0.1:40955] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 03:00:25,184 WARN [BP-885770418-188.40.62.62-1690599559790 heartbeating to localhost.localdomain/127.0.0.1:40955] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-885770418-188.40.62.62-1690599559790 (Datanode Uuid 2e608275-5f4d-4a8a-aa2d-0bc309941c30) service to localhost.localdomain/127.0.0.1:40955 2023-07-29 03:00:25,186 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686/dfs/data/data1/current/BP-885770418-188.40.62.62-1690599559790] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:00:25,187 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/cluster_4dabfe50-fbe4-76ac-3c8a-0d9ec4142686/dfs/data/data2/current/BP-885770418-188.40.62.62-1690599559790] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:00:25,190 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 03:00:25,202 INFO [Listener at localhost.localdomain/41871] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-07-29 03:00:25,321 INFO [Listener at localhost.localdomain/41871] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-29 03:00:25,338 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-29 03:00:25,345 INFO [Listener at localhost.localdomain/41871] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=65 (was 78), OpenFileDescriptor=435 (was 466), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=146 (was 259), ProcessCount=169 (was 169), AvailableMemoryMB=4525 (was 4823) 2023-07-29 03:00:25,352 INFO [Listener at localhost.localdomain/41871] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=66, OpenFileDescriptor=435, MaxFileDescriptor=60000, SystemLoadAverage=146, ProcessCount=169, AvailableMemoryMB=4525 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/hadoop.log.dir so I do NOT create it in target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058eb4d6-3cee-7a7f-640f-31e7e6c25996/hadoop.tmp.dir so I do NOT create it in target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13, deleteOnExit=true 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/test.cache.data in system properties and HBase conf 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/hadoop.tmp.dir in system properties and HBase conf 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/hadoop.log.dir in system properties and HBase conf 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-29 03:00:25,353 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-29 03:00:25,354 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-29 03:00:25,354 DEBUG [Listener at localhost.localdomain/41871] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-29 03:00:25,354 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-29 03:00:25,354 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-29 03:00:25,354 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-29 03:00:25,354 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 03:00:25,354 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-29 03:00:25,354 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/nfs.dump.dir in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/java.io.tmpdir in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-29 03:00:25,355 INFO [Listener at localhost.localdomain/41871] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-29 03:00:25,358 WARN [Listener at localhost.localdomain/41871] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 03:00:25,358 WARN [Listener at localhost.localdomain/41871] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 03:00:25,579 WARN [Listener at localhost.localdomain/41871] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 03:00:25,582 INFO [Listener at localhost.localdomain/41871] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 03:00:25,590 INFO [Listener at localhost.localdomain/41871] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/java.io.tmpdir/Jetty_localhost_localdomain_38747_hdfs____.713lbl/webapp 2023-07-29 03:00:25,660 INFO [Listener at localhost.localdomain/41871] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:38747 2023-07-29 03:00:25,662 WARN [Listener at localhost.localdomain/41871] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 03:00:25,662 WARN [Listener at localhost.localdomain/41871] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 03:00:25,838 WARN [Listener at localhost.localdomain/41579] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 03:00:25,851 WARN [Listener at localhost.localdomain/41579] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 03:00:25,853 WARN [Listener at localhost.localdomain/41579] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 03:00:25,854 INFO [Listener at localhost.localdomain/41579] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 03:00:25,858 INFO [Listener at localhost.localdomain/41579] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/java.io.tmpdir/Jetty_localhost_36203_datanode____.ole2zr/webapp 2023-07-29 03:00:25,931 INFO [Listener at localhost.localdomain/41579] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36203 2023-07-29 03:00:25,936 WARN [Listener at localhost.localdomain/40227] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 03:00:25,945 WARN [Listener at localhost.localdomain/40227] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 03:00:25,947 WARN [Listener at localhost.localdomain/40227] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 03:00:25,948 INFO [Listener at localhost.localdomain/40227] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 03:00:25,952 INFO [Listener at localhost.localdomain/40227] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/java.io.tmpdir/Jetty_localhost_45537_datanode____.yhvi01/webapp 2023-07-29 03:00:26,023 INFO [Listener at localhost.localdomain/40227] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45537 2023-07-29 03:00:26,029 WARN [Listener at localhost.localdomain/37169] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 03:00:26,034 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-29 03:00:26,574 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4e4d57d8a5edfa5: Processing first storage report for DS-5b4b3f87-948f-49b3-960e-6588d7f616bd from datanode 8fadec19-dd7b-4c7c-befb-4c17a3872ee7 2023-07-29 03:00:26,574 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4e4d57d8a5edfa5: from storage DS-5b4b3f87-948f-49b3-960e-6588d7f616bd node DatanodeRegistration(127.0.0.1:40221, datanodeUuid=8fadec19-dd7b-4c7c-befb-4c17a3872ee7, infoPort=45151, infoSecurePort=0, ipcPort=40227, storageInfo=lv=-57;cid=testClusterID;nsid=1077004566;c=1690599625360), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:00:26,574 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4e4d57d8a5edfa5: Processing first storage report for DS-89703ac4-467f-476f-a3e2-14b90f685491 from datanode 8fadec19-dd7b-4c7c-befb-4c17a3872ee7 2023-07-29 03:00:26,574 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4e4d57d8a5edfa5: from storage DS-89703ac4-467f-476f-a3e2-14b90f685491 node DatanodeRegistration(127.0.0.1:40221, datanodeUuid=8fadec19-dd7b-4c7c-befb-4c17a3872ee7, infoPort=45151, infoSecurePort=0, ipcPort=40227, storageInfo=lv=-57;cid=testClusterID;nsid=1077004566;c=1690599625360), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:00:26,655 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x636782f94851d82a: Processing first storage report for DS-e97e468f-b572-452c-96c7-167f05d1941f from datanode 0ecd9ce8-c6f6-4c5b-b972-168c0f2addc8 2023-07-29 03:00:26,656 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x636782f94851d82a: from storage DS-e97e468f-b572-452c-96c7-167f05d1941f node DatanodeRegistration(127.0.0.1:43217, datanodeUuid=0ecd9ce8-c6f6-4c5b-b972-168c0f2addc8, infoPort=46747, infoSecurePort=0, ipcPort=37169, storageInfo=lv=-57;cid=testClusterID;nsid=1077004566;c=1690599625360), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:00:26,656 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x636782f94851d82a: Processing first storage report for DS-261bbf76-0fba-45be-b883-41accd6c96e6 from datanode 0ecd9ce8-c6f6-4c5b-b972-168c0f2addc8 2023-07-29 03:00:26,656 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x636782f94851d82a: from storage DS-261bbf76-0fba-45be-b883-41accd6c96e6 node DatanodeRegistration(127.0.0.1:43217, datanodeUuid=0ecd9ce8-c6f6-4c5b-b972-168c0f2addc8, infoPort=46747, infoSecurePort=0, ipcPort=37169, storageInfo=lv=-57;cid=testClusterID;nsid=1077004566;c=1690599625360), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:00:26,744 DEBUG [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f 2023-07-29 03:00:26,749 INFO [Listener at localhost.localdomain/37169] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13/zookeeper_0, clientPort=57405, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-29 03:00:26,751 INFO [Listener at localhost.localdomain/37169] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57405 2023-07-29 03:00:26,751 INFO [Listener at localhost.localdomain/37169] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:26,752 INFO [Listener at localhost.localdomain/37169] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:26,767 INFO [Listener at localhost.localdomain/37169] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3 with version=8 2023-07-29 03:00:26,767 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/hbase-staging 2023-07-29 03:00:26,768 INFO [Listener at localhost.localdomain/37169] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 03:00:26,768 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:00:26,769 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 03:00:26,769 INFO [Listener at localhost.localdomain/37169] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 03:00:26,769 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:00:26,769 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 03:00:26,769 INFO [Listener at localhost.localdomain/37169] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 03:00:26,770 INFO [Listener at localhost.localdomain/37169] ipc.NettyRpcServer(120): Bind to /188.40.62.62:38955 2023-07-29 03:00:26,770 INFO [Listener at localhost.localdomain/37169] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:26,771 INFO [Listener at localhost.localdomain/37169] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:26,772 INFO [Listener at localhost.localdomain/37169] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38955 connecting to ZooKeeper ensemble=127.0.0.1:57405 2023-07-29 03:00:26,818 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:389550x0, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 03:00:26,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38955-0x101af0556410000 connected 2023-07-29 03:00:26,886 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 03:00:26,887 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:00:26,889 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 03:00:26,890 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38955 2023-07-29 03:00:26,890 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38955 2023-07-29 03:00:26,891 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38955 2023-07-29 03:00:26,892 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38955 2023-07-29 03:00:26,892 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38955 2023-07-29 03:00:26,892 INFO [Listener at localhost.localdomain/37169] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3, hbase.cluster.distributed=false 2023-07-29 03:00:26,908 INFO [Listener at localhost.localdomain/37169] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 03:00:26,909 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:00:26,909 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 03:00:26,909 INFO [Listener at localhost.localdomain/37169] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 03:00:26,909 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:00:26,909 INFO [Listener at localhost.localdomain/37169] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 03:00:26,909 INFO [Listener at localhost.localdomain/37169] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 03:00:26,911 INFO [Listener at localhost.localdomain/37169] ipc.NettyRpcServer(120): Bind to /188.40.62.62:45203 2023-07-29 03:00:26,911 INFO [Listener at localhost.localdomain/37169] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-29 03:00:26,912 DEBUG [Listener at localhost.localdomain/37169] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-29 03:00:26,912 INFO [Listener at localhost.localdomain/37169] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:26,913 INFO [Listener at localhost.localdomain/37169] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:26,914 INFO [Listener at localhost.localdomain/37169] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45203 connecting to ZooKeeper ensemble=127.0.0.1:57405 2023-07-29 03:00:26,924 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:452030x0, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 03:00:26,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45203-0x101af0556410001 connected 2023-07-29 03:00:26,925 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ZKUtil(164): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 03:00:26,926 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ZKUtil(164): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:00:26,927 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ZKUtil(164): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 03:00:26,927 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45203 2023-07-29 03:00:26,928 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45203 2023-07-29 03:00:26,928 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45203 2023-07-29 03:00:26,929 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45203 2023-07-29 03:00:26,929 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45203 2023-07-29 03:00:26,930 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:26,940 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 03:00:26,941 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:26,951 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 03:00:26,951 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 03:00:26,951 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:26,952 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 03:00:26,953 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,38955,1690599626768 from backup master directory 2023-07-29 03:00:26,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 03:00:26,960 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:26,960 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 03:00:26,960 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:26,960 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 03:00:26,974 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/hbase.id with ID: 5833e2e5-6f4b-4d8d-a613-48f707b211b2 2023-07-29 03:00:26,990 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:27,001 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:27,016 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7eb7d79d to 127.0.0.1:57405 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 03:00:27,027 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b718e1c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 03:00:27,027 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-29 03:00:27,028 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-29 03:00:27,028 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 03:00:27,030 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store-tmp 2023-07-29 03:00:27,045 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:27,045 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 03:00:27,045 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:27,045 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:27,045 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 03:00:27,045 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:27,045 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:00:27,045 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 03:00:27,046 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/WALs/jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:27,049 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C38955%2C1690599626768, suffix=, logDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/WALs/jenkins-hbase16.apache.org,38955,1690599626768, archiveDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/oldWALs, maxLogs=10 2023-07-29 03:00:27,067 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK] 2023-07-29 03:00:27,068 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK] 2023-07-29 03:00:27,070 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/WALs/jenkins-hbase16.apache.org,38955,1690599626768/jenkins-hbase16.apache.org%2C38955%2C1690599626768.1690599627050 2023-07-29 03:00:27,070 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK], DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK]] 2023-07-29 03:00:27,070 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-29 03:00:27,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:27,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:00:27,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:00:27,073 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:00:27,074 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-29 03:00:27,074 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-29 03:00:27,075 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:27,077 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:00:27,078 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:00:27,081 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:00:27,083 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 03:00:27,083 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=850788, jitterRate=0.08183367550373077}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 03:00:27,084 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 03:00:27,084 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-29 03:00:27,085 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-29 03:00:27,085 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-29 03:00:27,085 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-29 03:00:27,085 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-29 03:00:27,086 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-29 03:00:27,086 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-29 03:00:27,087 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-29 03:00:27,088 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-29 03:00:27,100 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-29 03:00:27,100 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-29 03:00:27,101 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-29 03:00:27,101 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-29 03:00:27,101 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-29 03:00:27,110 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:27,110 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-29 03:00:27,111 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-29 03:00:27,112 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-29 03:00:27,118 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 03:00:27,118 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 03:00:27,118 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:27,119 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase16.apache.org,38955,1690599626768, sessionid=0x101af0556410000, setting cluster-up flag (Was=false) 2023-07-29 03:00:27,135 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:27,160 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-29 03:00:27,163 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:27,182 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:27,210 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-29 03:00:27,214 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:27,216 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.hbase-snapshot/.tmp 2023-07-29 03:00:27,220 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-29 03:00:27,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:00:27,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:00:27,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:00:27,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:00:27,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-07-29 03:00:27,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 03:00:27,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,227 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690599657227 2023-07-29 03:00:27,227 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-29 03:00:27,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-29 03:00:27,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-29 03:00:27,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-29 03:00:27,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-29 03:00:27,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-29 03:00:27,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,229 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-29 03:00:27,229 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-29 03:00:27,229 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 03:00:27,229 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-29 03:00:27,229 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-29 03:00:27,229 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-29 03:00:27,229 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-29 03:00:27,229 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599627229,5,FailOnTimeoutGroup] 2023-07-29 03:00:27,231 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599627230,5,FailOnTimeoutGroup] 2023-07-29 03:00:27,231 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,231 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 03:00:27,231 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-29 03:00:27,231 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,231 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,232 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(951): ClusterId : 5833e2e5-6f4b-4d8d-a613-48f707b211b2 2023-07-29 03:00:27,233 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-29 03:00:27,242 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 03:00:27,243 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 03:00:27,243 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3 2023-07-29 03:00:27,244 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-29 03:00:27,244 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-29 03:00:27,254 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-29 03:00:27,256 DEBUG [RS:0;jenkins-hbase16:45203] zookeeper.ReadOnlyZKClient(139): Connect 0x42b142dc to 127.0.0.1:57405 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 03:00:27,259 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:27,265 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 03:00:27,266 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/info 2023-07-29 03:00:27,267 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 03:00:27,268 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:27,268 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 03:00:27,269 DEBUG [RS:0;jenkins-hbase16:45203] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4c76b4fa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 03:00:27,269 DEBUG [RS:0;jenkins-hbase16:45203] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@484929ac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 03:00:27,270 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/rep_barrier 2023-07-29 03:00:27,270 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 03:00:27,271 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:27,271 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 03:00:27,272 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/table 2023-07-29 03:00:27,273 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 03:00:27,273 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:27,274 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740 2023-07-29 03:00:27,275 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740 2023-07-29 03:00:27,277 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 03:00:27,278 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 03:00:27,279 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:45203 2023-07-29 03:00:27,279 INFO [RS:0;jenkins-hbase16:45203] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-29 03:00:27,279 INFO [RS:0;jenkins-hbase16:45203] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-29 03:00:27,279 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1022): About to register with Master. 2023-07-29 03:00:27,280 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase16.apache.org,38955,1690599626768 with isa=jenkins-hbase16.apache.org/188.40.62.62:45203, startcode=1690599626908 2023-07-29 03:00:27,280 DEBUG [RS:0;jenkins-hbase16:45203] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-29 03:00:27,280 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 03:00:27,281 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=776366, jitterRate=-0.01280057430267334}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 03:00:27,281 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 03:00:27,281 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 03:00:27,281 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 03:00:27,281 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 03:00:27,281 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 03:00:27,281 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 03:00:27,282 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 03:00:27,282 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 03:00:27,283 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:32815, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-29 03:00:27,284 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 03:00:27,284 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-29 03:00:27,284 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38955] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-29 03:00:27,285 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3 2023-07-29 03:00:27,285 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:41579 2023-07-29 03:00:27,285 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-29 03:00:27,286 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-29 03:00:27,288 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-29 03:00:27,293 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:00:27,293 DEBUG [RS:0;jenkins-hbase16:45203] zookeeper.ZKUtil(162): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,294 WARN [RS:0;jenkins-hbase16:45203] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 03:00:27,294 INFO [RS:0;jenkins-hbase16:45203] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 03:00:27,294 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,294 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,45203,1690599626908] 2023-07-29 03:00:27,298 DEBUG [RS:0;jenkins-hbase16:45203] zookeeper.ZKUtil(162): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,299 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-29 03:00:27,299 INFO [RS:0;jenkins-hbase16:45203] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-29 03:00:27,300 INFO [RS:0;jenkins-hbase16:45203] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-29 03:00:27,301 INFO [RS:0;jenkins-hbase16:45203] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-29 03:00:27,301 INFO [RS:0;jenkins-hbase16:45203] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,301 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-29 03:00:27,302 INFO [RS:0;jenkins-hbase16:45203] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,302 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,302 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,303 DEBUG [RS:0;jenkins-hbase16:45203] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:00:27,305 INFO [RS:0;jenkins-hbase16:45203] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,306 INFO [RS:0;jenkins-hbase16:45203] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,306 INFO [RS:0;jenkins-hbase16:45203] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,315 INFO [RS:0;jenkins-hbase16:45203] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-29 03:00:27,315 INFO [RS:0;jenkins-hbase16:45203] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,45203,1690599626908-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,324 INFO [RS:0;jenkins-hbase16:45203] regionserver.Replication(203): jenkins-hbase16.apache.org,45203,1690599626908 started 2023-07-29 03:00:27,324 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1637): Serving as jenkins-hbase16.apache.org,45203,1690599626908, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:45203, sessionid=0x101af0556410001 2023-07-29 03:00:27,324 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-29 03:00:27,324 DEBUG [RS:0;jenkins-hbase16:45203] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,324 DEBUG [RS:0;jenkins-hbase16:45203] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,45203,1690599626908' 2023-07-29 03:00:27,324 DEBUG [RS:0;jenkins-hbase16:45203] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 03:00:27,325 DEBUG [RS:0;jenkins-hbase16:45203] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 03:00:27,326 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-29 03:00:27,326 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-29 03:00:27,326 DEBUG [RS:0;jenkins-hbase16:45203] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,326 DEBUG [RS:0;jenkins-hbase16:45203] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,45203,1690599626908' 2023-07-29 03:00:27,326 DEBUG [RS:0;jenkins-hbase16:45203] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-29 03:00:27,326 DEBUG [RS:0;jenkins-hbase16:45203] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-29 03:00:27,326 DEBUG [RS:0;jenkins-hbase16:45203] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-29 03:00:27,326 INFO [RS:0;jenkins-hbase16:45203] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-29 03:00:27,326 INFO [RS:0;jenkins-hbase16:45203] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-29 03:00:27,429 INFO [RS:0;jenkins-hbase16:45203] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C45203%2C1690599626908, suffix=, logDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908, archiveDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/oldWALs, maxLogs=32 2023-07-29 03:00:27,438 DEBUG [jenkins-hbase16:38955] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-29 03:00:27,439 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,45203,1690599626908, state=OPENING 2023-07-29 03:00:27,446 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK] 2023-07-29 03:00:27,446 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK] 2023-07-29 03:00:27,448 INFO [RS:0;jenkins-hbase16:45203] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 2023-07-29 03:00:27,448 DEBUG [RS:0;jenkins-hbase16:45203] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK], DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK]] 2023-07-29 03:00:27,448 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-29 03:00:27,457 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:27,457 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,45203,1690599626908}] 2023-07-29 03:00:27,457 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 03:00:27,612 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,613 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-29 03:00:27,618 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:41564, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-29 03:00:27,627 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-29 03:00:27,627 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 03:00:27,629 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C45203%2C1690599626908.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908, archiveDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/oldWALs, maxLogs=32 2023-07-29 03:00:27,641 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK] 2023-07-29 03:00:27,642 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK] 2023-07-29 03:00:27,644 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.meta.1690599627629.meta 2023-07-29 03:00:27,644 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK], DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK]] 2023-07-29 03:00:27,644 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-29 03:00:27,644 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-29 03:00:27,644 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-29 03:00:27,644 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-29 03:00:27,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-29 03:00:27,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:27,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-29 03:00:27,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-29 03:00:27,646 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 03:00:27,648 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/info 2023-07-29 03:00:27,648 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/info 2023-07-29 03:00:27,648 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 03:00:27,649 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:27,649 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 03:00:27,650 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/rep_barrier 2023-07-29 03:00:27,650 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/rep_barrier 2023-07-29 03:00:27,651 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 03:00:27,651 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:27,652 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 03:00:27,653 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/table 2023-07-29 03:00:27,653 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/table 2023-07-29 03:00:27,653 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 03:00:27,654 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:27,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740 2023-07-29 03:00:27,657 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740 2023-07-29 03:00:27,659 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 03:00:27,660 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 03:00:27,661 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=836213, jitterRate=0.06330104172229767}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 03:00:27,661 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 03:00:27,662 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690599627612 2023-07-29 03:00:27,665 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-29 03:00:27,666 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-29 03:00:27,666 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,45203,1690599626908, state=OPEN 2023-07-29 03:00:27,676 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-29 03:00:27,676 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 03:00:27,679 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-29 03:00:27,679 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,45203,1690599626908 in 219 msec 2023-07-29 03:00:27,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-29 03:00:27,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 394 msec 2023-07-29 03:00:27,685 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 465 msec 2023-07-29 03:00:27,685 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690599627685, completionTime=-1 2023-07-29 03:00:27,685 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-07-29 03:00:27,685 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-29 03:00:27,688 DEBUG [hconnection-0x753cbdfb-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 03:00:27,690 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:41574, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 03:00:27,692 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-29 03:00:27,692 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690599687692 2023-07-29 03:00:27,692 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690599747692 2023-07-29 03:00:27,692 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 7 msec 2023-07-29 03:00:27,710 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,38955,1690599626768-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,710 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,38955,1690599626768-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,710 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,38955,1690599626768-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,711 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:38955, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,711 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-29 03:00:27,711 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-29 03:00:27,711 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 03:00:27,712 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-29 03:00:27,712 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-29 03:00:27,714 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-29 03:00:27,715 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-29 03:00:27,717 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:27,718 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b empty. 2023-07-29 03:00:27,719 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:27,719 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-29 03:00:27,737 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-29 03:00:27,738 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 569434e7c6b63c3e71b650f444f5403b, NAME => 'hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp 2023-07-29 03:00:27,750 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:27,750 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 569434e7c6b63c3e71b650f444f5403b, disabling compactions & flushes 2023-07-29 03:00:27,750 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:00:27,750 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:00:27,750 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. after waiting 0 ms 2023-07-29 03:00:27,750 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:00:27,750 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:00:27,750 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 569434e7c6b63c3e71b650f444f5403b: 2023-07-29 03:00:27,754 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-29 03:00:27,756 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599627756"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599627756"}]},"ts":"1690599627756"} 2023-07-29 03:00:27,759 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-29 03:00:27,761 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-29 03:00:27,761 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599627761"}]},"ts":"1690599627761"} 2023-07-29 03:00:27,763 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-29 03:00:27,808 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=569434e7c6b63c3e71b650f444f5403b, ASSIGN}] 2023-07-29 03:00:27,814 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=569434e7c6b63c3e71b650f444f5403b, ASSIGN 2023-07-29 03:00:27,816 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=569434e7c6b63c3e71b650f444f5403b, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,45203,1690599626908; forceNewPlan=false, retain=false 2023-07-29 03:00:27,968 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=569434e7c6b63c3e71b650f444f5403b, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:27,968 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599627968"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599627968"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599627968"}]},"ts":"1690599627968"} 2023-07-29 03:00:27,973 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 569434e7c6b63c3e71b650f444f5403b, server=jenkins-hbase16.apache.org,45203,1690599626908}] 2023-07-29 03:00:28,135 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:00:28,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 569434e7c6b63c3e71b650f444f5403b, NAME => 'hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b.', STARTKEY => '', ENDKEY => ''} 2023-07-29 03:00:28,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:28,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:28,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:28,137 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:28,139 INFO [StoreOpener-569434e7c6b63c3e71b650f444f5403b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:28,141 DEBUG [StoreOpener-569434e7c6b63c3e71b650f444f5403b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/info 2023-07-29 03:00:28,141 DEBUG [StoreOpener-569434e7c6b63c3e71b650f444f5403b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/info 2023-07-29 03:00:28,141 INFO [StoreOpener-569434e7c6b63c3e71b650f444f5403b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 569434e7c6b63c3e71b650f444f5403b columnFamilyName info 2023-07-29 03:00:28,142 INFO [StoreOpener-569434e7c6b63c3e71b650f444f5403b-1] regionserver.HStore(310): Store=569434e7c6b63c3e71b650f444f5403b/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:28,143 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:28,145 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:28,149 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:00:28,153 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 03:00:28,154 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 569434e7c6b63c3e71b650f444f5403b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=820251, jitterRate=0.04300317168235779}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 03:00:28,154 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 569434e7c6b63c3e71b650f444f5403b: 2023-07-29 03:00:28,155 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b., pid=6, masterSystemTime=1690599628126 2023-07-29 03:00:28,157 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:00:28,158 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:00:28,158 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=569434e7c6b63c3e71b650f444f5403b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:28,159 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599628158"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599628158"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599628158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599628158"}]},"ts":"1690599628158"} 2023-07-29 03:00:28,164 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-29 03:00:28,165 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 569434e7c6b63c3e71b650f444f5403b, server=jenkins-hbase16.apache.org,45203,1690599626908 in 188 msec 2023-07-29 03:00:28,169 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-29 03:00:28,170 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=569434e7c6b63c3e71b650f444f5403b, ASSIGN in 357 msec 2023-07-29 03:00:28,170 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-29 03:00:28,170 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599628170"}]},"ts":"1690599628170"} 2023-07-29 03:00:28,172 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-29 03:00:28,183 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-29 03:00:28,185 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 472 msec 2023-07-29 03:00:28,215 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-29 03:00:28,223 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-29 03:00:28,224 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:28,228 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-29 03:00:28,243 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 03:00:28,256 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 26 msec 2023-07-29 03:00:28,261 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-29 03:00:28,276 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 03:00:28,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 26 msec 2023-07-29 03:00:28,310 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-29 03:00:28,326 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-29 03:00:28,327 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.366sec 2023-07-29 03:00:28,327 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-29 03:00:28,327 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-29 03:00:28,327 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-29 03:00:28,327 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,38955,1690599626768-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-29 03:00:28,327 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,38955,1690599626768-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-29 03:00:28,331 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-29 03:00:28,333 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ReadOnlyZKClient(139): Connect 0x6415412a to 127.0.0.1:57405 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 03:00:28,345 DEBUG [Listener at localhost.localdomain/37169] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c5275e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 03:00:28,348 DEBUG [hconnection-0x7eeef07f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 03:00:28,351 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:55784, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 03:00:28,354 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:00:28,355 INFO [Listener at localhost.localdomain/37169] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:00:28,373 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-29 03:00:28,373 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:00:28,374 INFO [Listener at localhost.localdomain/37169] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-07-29 03:00:28,377 DEBUG [Listener at localhost.localdomain/37169] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-29 03:00:28,380 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:36094, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-29 03:00:28,382 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38955] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-07-29 03:00:28,382 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38955] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-07-29 03:00:28,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38955] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-29 03:00:28,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38955] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-07-29 03:00:28,388 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-07-29 03:00:28,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38955] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-07-29 03:00:28,390 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-29 03:00:28,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38955] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-29 03:00:28,392 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,392 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0 empty. 2023-07-29 03:00:28,393 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,393 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-07-29 03:00:28,405 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-07-29 03:00:28,406 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6edf31243051b9aa72f064147bfc47d0, NAME => 'TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/.tmp 2023-07-29 03:00:28,414 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:28,414 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing 6edf31243051b9aa72f064147bfc47d0, disabling compactions & flushes 2023-07-29 03:00:28,414 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:28,414 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:28,414 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. after waiting 0 ms 2023-07-29 03:00:28,414 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:28,414 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:28,414 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:28,417 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-07-29 03:00:28,419 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690599628419"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599628419"}]},"ts":"1690599628419"} 2023-07-29 03:00:28,421 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-29 03:00:28,421 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-29 03:00:28,422 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599628422"}]},"ts":"1690599628422"} 2023-07-29 03:00:28,423 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-07-29 03:00:28,440 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=6edf31243051b9aa72f064147bfc47d0, ASSIGN}] 2023-07-29 03:00:28,443 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=6edf31243051b9aa72f064147bfc47d0, ASSIGN 2023-07-29 03:00:28,444 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=6edf31243051b9aa72f064147bfc47d0, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,45203,1690599626908; forceNewPlan=false, retain=false 2023-07-29 03:00:28,597 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6edf31243051b9aa72f064147bfc47d0, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:28,597 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690599628597"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599628597"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599628597"}]},"ts":"1690599628597"} 2023-07-29 03:00:28,603 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 6edf31243051b9aa72f064147bfc47d0, server=jenkins-hbase16.apache.org,45203,1690599626908}] 2023-07-29 03:00:28,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:28,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6edf31243051b9aa72f064147bfc47d0, NAME => 'TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.', STARTKEY => '', ENDKEY => ''} 2023-07-29 03:00:28,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:28,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,771 INFO [StoreOpener-6edf31243051b9aa72f064147bfc47d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,773 DEBUG [StoreOpener-6edf31243051b9aa72f064147bfc47d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info 2023-07-29 03:00:28,774 DEBUG [StoreOpener-6edf31243051b9aa72f064147bfc47d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info 2023-07-29 03:00:28,774 INFO [StoreOpener-6edf31243051b9aa72f064147bfc47d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6edf31243051b9aa72f064147bfc47d0 columnFamilyName info 2023-07-29 03:00:28,775 INFO [StoreOpener-6edf31243051b9aa72f064147bfc47d0-1] regionserver.HStore(310): Store=6edf31243051b9aa72f064147bfc47d0/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:28,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:28,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 03:00:28,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 6edf31243051b9aa72f064147bfc47d0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=771103, jitterRate=-0.01949244737625122}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 03:00:28,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:28,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0., pid=11, masterSystemTime=1690599628759 2023-07-29 03:00:28,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:28,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:28,793 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6edf31243051b9aa72f064147bfc47d0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:28,793 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690599628793"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599628793"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599628793"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599628793"}]},"ts":"1690599628793"} 2023-07-29 03:00:28,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-29 03:00:28,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 6edf31243051b9aa72f064147bfc47d0, server=jenkins-hbase16.apache.org,45203,1690599626908 in 192 msec 2023-07-29 03:00:28,800 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-29 03:00:28,801 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=6edf31243051b9aa72f064147bfc47d0, ASSIGN in 358 msec 2023-07-29 03:00:28,802 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-29 03:00:28,802 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599628802"}]},"ts":"1690599628802"} 2023-07-29 03:00:28,803 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-07-29 03:00:28,810 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-07-29 03:00:28,812 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 429 msec 2023-07-29 03:00:30,028 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-29 03:00:33,299 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-29 03:00:33,300 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-29 03:00:33,301 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-07-29 03:00:35,190 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 03:00:35,190 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-29 03:00:38,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38955] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-29 03:00:38,393 INFO [Listener at localhost.localdomain/37169] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-07-29 03:00:38,398 DEBUG [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-07-29 03:00:38,399 DEBUG [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:38,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:38,420 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 6edf31243051b9aa72f064147bfc47d0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:00:38,436 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/1c7b910747a746529532ef3b66fff66b 2023-07-29 03:00:38,446 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/1c7b910747a746529532ef3b66fff66b as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/1c7b910747a746529532ef3b66fff66b 2023-07-29 03:00:38,453 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/1c7b910747a746529532ef3b66fff66b, entries=8, sequenceid=12, filesize=13.0 K 2023-07-29 03:00:38,454 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=19.50 KB/19969 for 6edf31243051b9aa72f064147bfc47d0 in 34ms, sequenceid=12, compaction requested=false 2023-07-29 03:00:38,455 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:38,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:38,456 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 6edf31243051b9aa72f064147bfc47d0 1/1 column families, dataSize=20.53 KB heapSize=22.28 KB 2023-07-29 03:00:38,471 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=20.53 KB at sequenceid=35 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/31d3e090a00f4746a5abc1179cc3195f 2023-07-29 03:00:38,477 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/31d3e090a00f4746a5abc1179cc3195f as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f 2023-07-29 03:00:38,483 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f, entries=20, sequenceid=35, filesize=25.3 K 2023-07-29 03:00:38,484 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~20.53 KB/21020, heapSize ~22.27 KB/22800, currentSize=4.11 KB/4204 for 6edf31243051b9aa72f064147bfc47d0 in 27ms, sequenceid=35, compaction requested=false 2023-07-29 03:00:38,484 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:38,484 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=38.3 K, sizeToCheck=16.0 K 2023-07-29 03:00:38,484 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-07-29 03:00:38,485 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f because midkey is the same as first or last row 2023-07-29 03:00:40,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:40,480 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 6edf31243051b9aa72f064147bfc47d0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:00:40,500 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=46 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/b5365ba69aef4657899410f937158b72 2023-07-29 03:00:40,508 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/b5365ba69aef4657899410f937158b72 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/b5365ba69aef4657899410f937158b72 2023-07-29 03:00:40,514 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/b5365ba69aef4657899410f937158b72, entries=8, sequenceid=46, filesize=13.0 K 2023-07-29 03:00:40,515 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=19.50 KB/19969 for 6edf31243051b9aa72f064147bfc47d0 in 35ms, sequenceid=46, compaction requested=true 2023-07-29 03:00:40,515 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:40,515 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=51.2 K, sizeToCheck=16.0 K 2023-07-29 03:00:40,515 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-07-29 03:00:40,516 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f because midkey is the same as first or last row 2023-07-29 03:00:40,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:40,517 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:40,517 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:00:40,518 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 6edf31243051b9aa72f064147bfc47d0 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-07-29 03:00:40,519 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 52472 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:00:40,520 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): 6edf31243051b9aa72f064147bfc47d0/info is initiating minor compaction (all files) 2023-07-29 03:00:40,520 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6edf31243051b9aa72f064147bfc47d0/info in TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:40,520 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/1c7b910747a746529532ef3b66fff66b, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/b5365ba69aef4657899410f937158b72] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp, totalSize=51.2 K 2023-07-29 03:00:40,521 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 1c7b910747a746529532ef3b66fff66b, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1690599638405 2023-07-29 03:00:40,521 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 31d3e090a00f4746a5abc1179cc3195f, keycount=20, bloomtype=ROW, size=25.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1690599638422 2023-07-29 03:00:40,522 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting b5365ba69aef4657899410f937158b72, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690599638456 2023-07-29 03:00:40,535 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=70 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/57697bcc9867438a95aa3468d5cf4726 2023-07-29 03:00:40,543 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/57697bcc9867438a95aa3468d5cf4726 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/57697bcc9867438a95aa3468d5cf4726 2023-07-29 03:00:40,543 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6edf31243051b9aa72f064147bfc47d0#info#compaction#19 average throughput is 18.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:00:40,555 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/57697bcc9867438a95aa3468d5cf4726, entries=21, sequenceid=70, filesize=26.3 K 2023-07-29 03:00:40,557 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=7.18 KB/7357 for 6edf31243051b9aa72f064147bfc47d0 in 40ms, sequenceid=70, compaction requested=false 2023-07-29 03:00:40,557 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:40,557 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=77.6 K, sizeToCheck=16.0 K 2023-07-29 03:00:40,557 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-07-29 03:00:40,557 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/57697bcc9867438a95aa3468d5cf4726 because midkey is the same as first or last row 2023-07-29 03:00:40,562 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/709cfcc6b0d14454824834e3075deef9 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9 2023-07-29 03:00:40,569 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 6edf31243051b9aa72f064147bfc47d0/info of 6edf31243051b9aa72f064147bfc47d0 into 709cfcc6b0d14454824834e3075deef9(size=41.9 K), total size for store is 68.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:00:40,570 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:40,570 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0., storeName=6edf31243051b9aa72f064147bfc47d0/info, priority=13, startTime=1690599640517; duration=0sec 2023-07-29 03:00:40,571 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=68.2 K, sizeToCheck=16.0 K 2023-07-29 03:00:40,571 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-07-29 03:00:40,571 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9 because midkey is the same as first or last row 2023-07-29 03:00:40,571 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:42,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,543 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 6edf31243051b9aa72f064147bfc47d0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:00:42,561 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/2de100fe4c2e4f5a8fce6a7992b332f8 2023-07-29 03:00:42,569 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/2de100fe4c2e4f5a8fce6a7992b332f8 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/2de100fe4c2e4f5a8fce6a7992b332f8 2023-07-29 03:00:42,575 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/2de100fe4c2e4f5a8fce6a7992b332f8, entries=8, sequenceid=82, filesize=13.0 K 2023-07-29 03:00:42,576 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=18.47 KB/18918 for 6edf31243051b9aa72f064147bfc47d0 in 34ms, sequenceid=82, compaction requested=true 2023-07-29 03:00:42,576 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:42,576 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=81.2 K, sizeToCheck=16.0 K 2023-07-29 03:00:42,576 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-07-29 03:00:42,576 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9 because midkey is the same as first or last row 2023-07-29 03:00:42,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,577 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:42,577 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:00:42,577 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 6edf31243051b9aa72f064147bfc47d0 1/1 column families, dataSize=19.50 KB heapSize=21.18 KB 2023-07-29 03:00:42,579 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 83139 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:00:42,579 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): 6edf31243051b9aa72f064147bfc47d0/info is initiating minor compaction (all files) 2023-07-29 03:00:42,579 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6edf31243051b9aa72f064147bfc47d0/info in TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:42,579 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/57697bcc9867438a95aa3468d5cf4726, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/2de100fe4c2e4f5a8fce6a7992b332f8] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp, totalSize=81.2 K 2023-07-29 03:00:42,579 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 709cfcc6b0d14454824834e3075deef9, keycount=36, bloomtype=ROW, size=41.9 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690599638405 2023-07-29 03:00:42,580 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 57697bcc9867438a95aa3468d5cf4726, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=70, earliestPutTs=1690599640480 2023-07-29 03:00:42,580 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 2de100fe4c2e4f5a8fce6a7992b332f8, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690599640519 2023-07-29 03:00:42,593 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=19.50 KB at sequenceid=104 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/583d9e68e9bf47b98e881ca39c456150 2023-07-29 03:00:42,595 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=6edf31243051b9aa72f064147bfc47d0, server=jenkins-hbase16.apache.org,45203,1690599626908 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-29 03:00:42,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] ipc.CallRunner(144): callId: 103 service: ClientService methodName: Mutate size: 1.1 K connection: 188.40.62.62:55784 deadline: 1690599652595, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=6edf31243051b9aa72f064147bfc47d0, server=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:42,599 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6edf31243051b9aa72f064147bfc47d0#info#compaction#22 average throughput is 16.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:00:42,601 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/583d9e68e9bf47b98e881ca39c456150 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/583d9e68e9bf47b98e881ca39c456150 2023-07-29 03:00:42,612 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/583d9e68e9bf47b98e881ca39c456150, entries=19, sequenceid=104, filesize=24.3 K 2023-07-29 03:00:42,613 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~19.50 KB/19969, heapSize ~21.16 KB/21672, currentSize=10.26 KB/10510 for 6edf31243051b9aa72f064147bfc47d0 in 36ms, sequenceid=104, compaction requested=false 2023-07-29 03:00:42,613 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:42,613 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=105.5 K, sizeToCheck=16.0 K 2023-07-29 03:00:42,613 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-07-29 03:00:42,613 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9 because midkey is the same as first or last row 2023-07-29 03:00:42,618 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/7b5d7d6008514f1b856081458363714a as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a 2023-07-29 03:00:42,626 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 6edf31243051b9aa72f064147bfc47d0/info of 6edf31243051b9aa72f064147bfc47d0 into 7b5d7d6008514f1b856081458363714a(size=71.9 K), total size for store is 96.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:00:42,626 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:42,626 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0., storeName=6edf31243051b9aa72f064147bfc47d0/info, priority=13, startTime=1690599642576; duration=0sec 2023-07-29 03:00:42,626 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=96.2 K, sizeToCheck=16.0 K 2023-07-29 03:00:42,626 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-07-29 03:00:42,627 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:42,627 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:42,628 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38955] assignment.AssignmentManager(1140): Split request from jenkins-hbase16.apache.org,45203,1690599626908, parent={ENCODED => 6edf31243051b9aa72f064147bfc47d0, NAME => 'TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-07-29 03:00:42,634 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38955] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:42,639 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38955] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=6edf31243051b9aa72f064147bfc47d0, daughterA=1335e692f5ac52c1fcc86a408e0ec39c, daughterB=ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:42,640 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=6edf31243051b9aa72f064147bfc47d0, daughterA=1335e692f5ac52c1fcc86a408e0ec39c, daughterB=ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:42,640 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=6edf31243051b9aa72f064147bfc47d0, daughterA=1335e692f5ac52c1fcc86a408e0ec39c, daughterB=ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:42,640 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=6edf31243051b9aa72f064147bfc47d0, daughterA=1335e692f5ac52c1fcc86a408e0ec39c, daughterB=ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:42,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=6edf31243051b9aa72f064147bfc47d0, UNASSIGN}] 2023-07-29 03:00:42,650 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=6edf31243051b9aa72f064147bfc47d0, UNASSIGN 2023-07-29 03:00:42,651 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6edf31243051b9aa72f064147bfc47d0, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:42,651 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690599642650"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599642650"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599642650"}]},"ts":"1690599642650"} 2023-07-29 03:00:42,652 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 6edf31243051b9aa72f064147bfc47d0, server=jenkins-hbase16.apache.org,45203,1690599626908}] 2023-07-29 03:00:42,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 6edf31243051b9aa72f064147bfc47d0, disabling compactions & flushes 2023-07-29 03:00:42,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:42,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:42,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. after waiting 0 ms 2023-07-29 03:00:42,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:42,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 6edf31243051b9aa72f064147bfc47d0 1/1 column families, dataSize=10.26 KB heapSize=11.27 KB 2023-07-29 03:00:42,831 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=10.26 KB at sequenceid=118 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/14c40be58689445ea139a37e6985c220 2023-07-29 03:00:42,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.tmp/info/14c40be58689445ea139a37e6985c220 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/14c40be58689445ea139a37e6985c220 2023-07-29 03:00:42,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/14c40be58689445ea139a37e6985c220, entries=10, sequenceid=118, filesize=15.0 K 2023-07-29 03:00:42,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~10.26 KB/10510, heapSize ~11.25 KB/11520, currentSize=0 B/0 for 6edf31243051b9aa72f064147bfc47d0 in 31ms, sequenceid=118, compaction requested=true 2023-07-29 03:00:42,855 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/1c7b910747a746529532ef3b66fff66b, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/b5365ba69aef4657899410f937158b72, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/57697bcc9867438a95aa3468d5cf4726, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/2de100fe4c2e4f5a8fce6a7992b332f8] to archive 2023-07-29 03:00:42,857 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-29 03:00:42,860 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/1c7b910747a746529532ef3b66fff66b to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/1c7b910747a746529532ef3b66fff66b 2023-07-29 03:00:42,862 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/31d3e090a00f4746a5abc1179cc3195f 2023-07-29 03:00:42,865 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/709cfcc6b0d14454824834e3075deef9 2023-07-29 03:00:42,867 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/b5365ba69aef4657899410f937158b72 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/b5365ba69aef4657899410f937158b72 2023-07-29 03:00:42,869 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/57697bcc9867438a95aa3468d5cf4726 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/57697bcc9867438a95aa3468d5cf4726 2023-07-29 03:00:42,870 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/2de100fe4c2e4f5a8fce6a7992b332f8 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/2de100fe4c2e4f5a8fce6a7992b332f8 2023-07-29 03:00:42,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/recovered.edits/121.seqid, newMaxSeqId=121, maxSeqId=1 2023-07-29 03:00:42,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. 2023-07-29 03:00:42,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 6edf31243051b9aa72f064147bfc47d0: 2023-07-29 03:00:42,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,882 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6edf31243051b9aa72f064147bfc47d0, regionState=CLOSED 2023-07-29 03:00:42,883 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690599642882"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599642882"}]},"ts":"1690599642882"} 2023-07-29 03:00:42,886 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-29 03:00:42,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 6edf31243051b9aa72f064147bfc47d0, server=jenkins-hbase16.apache.org,45203,1690599626908 in 232 msec 2023-07-29 03:00:42,888 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-29 03:00:42,888 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=6edf31243051b9aa72f064147bfc47d0, UNASSIGN in 239 msec 2023-07-29 03:00:42,903 INFO [PEWorker-5] assignment.SplitTableRegionProcedure(694): pid=12 splitting 3 storefiles, region=6edf31243051b9aa72f064147bfc47d0, threads=3 2023-07-29 03:00:42,904 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/14c40be58689445ea139a37e6985c220 for region: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,904 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/583d9e68e9bf47b98e881ca39c456150 for region: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,904 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a for region: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,917 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/14c40be58689445ea139a37e6985c220, top=true 2023-07-29 03:00:42,917 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/583d9e68e9bf47b98e881ca39c456150, top=true 2023-07-29 03:00:42,927 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.splits/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-14c40be58689445ea139a37e6985c220 for child: ad0deeb09f0cf4eec5a424c810de0bf1, parent: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,928 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/14c40be58689445ea139a37e6985c220 for region: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,928 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/.splits/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-583d9e68e9bf47b98e881ca39c456150 for child: ad0deeb09f0cf4eec5a424c810de0bf1, parent: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,928 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/583d9e68e9bf47b98e881ca39c456150 for region: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,942 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a for region: 6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:00:42,943 DEBUG [PEWorker-5] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 6edf31243051b9aa72f064147bfc47d0 Daughter A: 1 storefiles, Daughter B: 3 storefiles. 2023-07-29 03:00:42,974 DEBUG [PEWorker-5] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/recovered.edits/121.seqid, newMaxSeqId=121, maxSeqId=-1 2023-07-29 03:00:42,976 DEBUG [PEWorker-5] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/recovered.edits/121.seqid, newMaxSeqId=121, maxSeqId=-1 2023-07-29 03:00:42,978 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690599642978"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1690599642978"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1690599642978"}]},"ts":"1690599642978"} 2023-07-29 03:00:42,978 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690599642978"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599642978"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599642978"}]},"ts":"1690599642978"} 2023-07-29 03:00:42,978 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690599642978"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599642978"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599642978"}]},"ts":"1690599642978"} 2023-07-29 03:00:43,012 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45203] regionserver.HRegion(9158): Flush requested on 1588230740 2023-07-29 03:00:43,012 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-07-29 03:00:43,012 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-07-29 03:00:43,022 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1335e692f5ac52c1fcc86a408e0ec39c, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad0deeb09f0cf4eec5a424c810de0bf1, ASSIGN}] 2023-07-29 03:00:43,025 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1335e692f5ac52c1fcc86a408e0ec39c, ASSIGN 2023-07-29 03:00:43,025 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad0deeb09f0cf4eec5a424c810de0bf1, ASSIGN 2023-07-29 03:00:43,026 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1335e692f5ac52c1fcc86a408e0ec39c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,45203,1690599626908; forceNewPlan=false, retain=false 2023-07-29 03:00:43,026 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad0deeb09f0cf4eec5a424c810de0bf1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,45203,1690599626908; forceNewPlan=false, retain=false 2023-07-29 03:00:43,179 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1335e692f5ac52c1fcc86a408e0ec39c, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:43,179 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=ad0deeb09f0cf4eec5a424c810de0bf1, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:43,179 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690599643179"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599643179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599643179"}]},"ts":"1690599643179"} 2023-07-29 03:00:43,179 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690599643179"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599643179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599643179"}]},"ts":"1690599643179"} 2023-07-29 03:00:43,183 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 1335e692f5ac52c1fcc86a408e0ec39c, server=jenkins-hbase16.apache.org,45203,1690599626908}] 2023-07-29 03:00:43,186 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908}] 2023-07-29 03:00:43,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:00:43,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ad0deeb09f0cf4eec5a424c810de0bf1, NAME => 'TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.', STARTKEY => 'row0064', ENDKEY => ''} 2023-07-29 03:00:43,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:43,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:43,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:43,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:43,345 INFO [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:43,347 DEBUG [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info 2023-07-29 03:00:43,347 DEBUG [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info 2023-07-29 03:00:43,347 INFO [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad0deeb09f0cf4eec5a424c810de0bf1 columnFamilyName info 2023-07-29 03:00:43,362 DEBUG [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0->hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a-top 2023-07-29 03:00:43,368 DEBUG [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-14c40be58689445ea139a37e6985c220 2023-07-29 03:00:43,374 DEBUG [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-583d9e68e9bf47b98e881ca39c456150 2023-07-29 03:00:43,374 INFO [StoreOpener-ad0deeb09f0cf4eec5a424c810de0bf1-1] regionserver.HStore(310): Store=ad0deeb09f0cf4eec5a424c810de0bf1/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:43,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:43,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:43,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:00:43,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened ad0deeb09f0cf4eec5a424c810de0bf1; next sequenceid=122; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=810908, jitterRate=0.031123608350753784}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 03:00:43,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:00:43,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., pid=18, masterSystemTime=1690599643337 2023-07-29 03:00:43,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:43,383 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:00:43,385 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:00:43,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:00:43,385 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): ad0deeb09f0cf4eec5a424c810de0bf1/info is initiating minor compaction (all files) 2023-07-29 03:00:43,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:00:43,385 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad0deeb09f0cf4eec5a424c810de0bf1/info in TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:00:43,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:00:43,385 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0->hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a-top, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-583d9e68e9bf47b98e881ca39c456150, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-14c40be58689445ea139a37e6985c220] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp, totalSize=111.2 K 2023-07-29 03:00:43,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1335e692f5ac52c1fcc86a408e0ec39c, NAME => 'TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.', STARTKEY => '', ENDKEY => 'row0064'} 2023-07-29 03:00:43,385 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=ad0deeb09f0cf4eec5a424c810de0bf1, regionState=OPEN, openSeqNum=122, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:43,387 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690599643385"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599643385"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599643385"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599643385"}]},"ts":"1690599643385"} 2023-07-29 03:00:43,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:00:43,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:00:43,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:00:43,387 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0, keycount=32, bloomtype=ROW, size=71.9 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690599638405 2023-07-29 03:00:43,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:00:43,388 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-583d9e68e9bf47b98e881ca39c456150, keycount=19, bloomtype=ROW, size=24.3 K, encoding=NONE, compression=NONE, seqNum=104, earliestPutTs=1690599642544 2023-07-29 03:00:43,389 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-14c40be58689445ea139a37e6985c220, keycount=10, bloomtype=ROW, size=15.0 K, encoding=NONE, compression=NONE, seqNum=118, earliestPutTs=1690599642578 2023-07-29 03:00:43,389 INFO [StoreOpener-1335e692f5ac52c1fcc86a408e0ec39c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:00:43,390 DEBUG [StoreOpener-1335e692f5ac52c1fcc86a408e0ec39c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info 2023-07-29 03:00:43,391 DEBUG [StoreOpener-1335e692f5ac52c1fcc86a408e0ec39c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info 2023-07-29 03:00:43,391 INFO [StoreOpener-1335e692f5ac52c1fcc86a408e0ec39c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1335e692f5ac52c1fcc86a408e0ec39c columnFamilyName info 2023-07-29 03:00:43,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-07-29 03:00:43,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908 in 203 msec 2023-07-29 03:00:43,394 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad0deeb09f0cf4eec5a424c810de0bf1, ASSIGN in 370 msec 2023-07-29 03:00:43,403 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad0deeb09f0cf4eec5a424c810de0bf1#info#compaction#25 average throughput is 31.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:00:43,408 DEBUG [StoreOpener-1335e692f5ac52c1fcc86a408e0ec39c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0->hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a-bottom 2023-07-29 03:00:43,409 INFO [StoreOpener-1335e692f5ac52c1fcc86a408e0ec39c-1] regionserver.HStore(310): Store=1335e692f5ac52c1fcc86a408e0ec39c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:00:43,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:00:43,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:00:43,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:00:43,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1335e692f5ac52c1fcc86a408e0ec39c; next sequenceid=122; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=873032, jitterRate=0.1101183295249939}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 03:00:43,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1335e692f5ac52c1fcc86a408e0ec39c: 2023-07-29 03:00:43,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c., pid=17, masterSystemTime=1690599643337 2023-07-29 03:00:43,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:43,423 DEBUG [RS:0;jenkins-hbase16:45203-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-29 03:00:43,424 INFO [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:00:43,424 DEBUG [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.HStore(1912): 1335e692f5ac52c1fcc86a408e0ec39c/info is initiating minor compaction (all files) 2023-07-29 03:00:43,424 INFO [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1335e692f5ac52c1fcc86a408e0ec39c/info in TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:00:43,425 INFO [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0->hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a-bottom] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/.tmp, totalSize=71.9 K 2023-07-29 03:00:43,425 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/e51dda86136a445e8ffbf3cb09b1d591 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/e51dda86136a445e8ffbf3cb09b1d591 2023-07-29 03:00:43,425 DEBUG [RS:0;jenkins-hbase16:45203-longCompactions-0] compactions.Compactor(207): Compacting 7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0, keycount=32, bloomtype=ROW, size=71.9 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690599638405 2023-07-29 03:00:43,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:00:43,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:00:43,426 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1335e692f5ac52c1fcc86a408e0ec39c, regionState=OPEN, openSeqNum=122, regionLocation=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:00:43,426 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690599643426"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599643426"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599643426"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599643426"}]},"ts":"1690599643426"} 2023-07-29 03:00:43,428 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/.tmp/info/97642ad5e16a45a88e18130ed1bcb56c 2023-07-29 03:00:43,432 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-29 03:00:43,432 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 1335e692f5ac52c1fcc86a408e0ec39c, server=jenkins-hbase16.apache.org,45203,1690599626908 in 246 msec 2023-07-29 03:00:43,436 INFO [RS:0;jenkins-hbase16:45203-longCompactions-0] throttle.PressureAwareThroughputController(145): 1335e692f5ac52c1fcc86a408e0ec39c#info#compaction#26 average throughput is 15.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:00:43,436 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-07-29 03:00:43,436 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1335e692f5ac52c1fcc86a408e0ec39c, ASSIGN in 410 msec 2023-07-29 03:00:43,438 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ad0deeb09f0cf4eec5a424c810de0bf1/info of ad0deeb09f0cf4eec5a424c810de0bf1 into e51dda86136a445e8ffbf3cb09b1d591(size=36.9 K), total size for store is 36.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:00:43,439 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:00:43,439 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., storeName=ad0deeb09f0cf4eec5a424c810de0bf1/info, priority=13, startTime=1690599643382; duration=0sec 2023-07-29 03:00:43,439 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:43,439 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=6edf31243051b9aa72f064147bfc47d0, daughterA=1335e692f5ac52c1fcc86a408e0ec39c, daughterB=ad0deeb09f0cf4eec5a424c810de0bf1 in 802 msec 2023-07-29 03:00:43,457 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/.tmp/table/df8eaa97e55643f2a95a1d8c28400858 2023-07-29 03:00:43,459 DEBUG [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/.tmp/info/e380bbe5d02d4f4ca404e5297dcb115f as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info/e380bbe5d02d4f4ca404e5297dcb115f 2023-07-29 03:00:43,464 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/.tmp/info/97642ad5e16a45a88e18130ed1bcb56c as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/info/97642ad5e16a45a88e18130ed1bcb56c 2023-07-29 03:00:43,466 INFO [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in 1335e692f5ac52c1fcc86a408e0ec39c/info of 1335e692f5ac52c1fcc86a408e0ec39c into e380bbe5d02d4f4ca404e5297dcb115f(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:00:43,466 DEBUG [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1335e692f5ac52c1fcc86a408e0ec39c: 2023-07-29 03:00:43,466 INFO [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c., storeName=1335e692f5ac52c1fcc86a408e0ec39c/info, priority=15, startTime=1690599643423; duration=0sec 2023-07-29 03:00:43,466 DEBUG [RS:0;jenkins-hbase16:45203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:00:43,470 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/info/97642ad5e16a45a88e18130ed1bcb56c, entries=29, sequenceid=17, filesize=8.6 K 2023-07-29 03:00:43,471 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/.tmp/table/df8eaa97e55643f2a95a1d8c28400858 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/table/df8eaa97e55643f2a95a1d8c28400858 2023-07-29 03:00:43,478 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/table/df8eaa97e55643f2a95a1d8c28400858, entries=4, sequenceid=17, filesize=4.8 K 2023-07-29 03:00:43,479 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4939, heapSize ~8.08 KB/8272, currentSize=2.26 KB/2316 for 1588230740 in 467ms, sequenceid=17, compaction requested=false 2023-07-29 03:00:43,480 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-07-29 03:00:48,473 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-29 03:00:52,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] ipc.CallRunner(144): callId: 105 service: ClientService methodName: Mutate size: 1.1 K connection: 188.40.62.62:55784 deadline: 1690599662651, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1690599628382.6edf31243051b9aa72f064147bfc47d0. is not online on jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:01:14,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:14,780 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:01:14,798 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=133 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/42269e43617c414fb82459adaa508461 2023-07-29 03:01:14,805 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/42269e43617c414fb82459adaa508461 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/42269e43617c414fb82459adaa508461 2023-07-29 03:01:14,813 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/42269e43617c414fb82459adaa508461, entries=8, sequenceid=133, filesize=13.0 K 2023-07-29 03:01:14,814 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=18.47 KB/18918 for ad0deeb09f0cf4eec5a424c810de0bf1 in 33ms, sequenceid=133, compaction requested=false 2023-07-29 03:01:14,814 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:14,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:14,815 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=19.50 KB heapSize=21.18 KB 2023-07-29 03:01:14,832 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=19.50 KB at sequenceid=155 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/035ee277d75f4e6e9c226ebeb40b2a9e 2023-07-29 03:01:14,839 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/035ee277d75f4e6e9c226ebeb40b2a9e as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/035ee277d75f4e6e9c226ebeb40b2a9e 2023-07-29 03:01:14,844 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/035ee277d75f4e6e9c226ebeb40b2a9e, entries=19, sequenceid=155, filesize=24.3 K 2023-07-29 03:01:14,845 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~19.50 KB/19969, heapSize ~21.16 KB/21672, currentSize=7.18 KB/7357 for ad0deeb09f0cf4eec5a424c810de0bf1 in 31ms, sequenceid=155, compaction requested=true 2023-07-29 03:01:14,845 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:14,846 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:14,846 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:01:14,847 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 75955 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:01:14,847 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): ad0deeb09f0cf4eec5a424c810de0bf1/info is initiating minor compaction (all files) 2023-07-29 03:01:14,847 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad0deeb09f0cf4eec5a424c810de0bf1/info in TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:14,847 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/e51dda86136a445e8ffbf3cb09b1d591, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/42269e43617c414fb82459adaa508461, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/035ee277d75f4e6e9c226ebeb40b2a9e] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp, totalSize=74.2 K 2023-07-29 03:01:14,848 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting e51dda86136a445e8ffbf3cb09b1d591, keycount=31, bloomtype=ROW, size=36.9 K, encoding=NONE, compression=NONE, seqNum=118, earliestPutTs=1690599640535 2023-07-29 03:01:14,848 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 42269e43617c414fb82459adaa508461, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=133, earliestPutTs=1690599672763 2023-07-29 03:01:14,849 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 035ee277d75f4e6e9c226ebeb40b2a9e, keycount=19, bloomtype=ROW, size=24.3 K, encoding=NONE, compression=NONE, seqNum=155, earliestPutTs=1690599674781 2023-07-29 03:01:14,861 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad0deeb09f0cf4eec5a424c810de0bf1#info#compaction#30 average throughput is 29.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:01:14,882 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/76b0c38203434b26a3d4f556ae3206f3 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/76b0c38203434b26a3d4f556ae3206f3 2023-07-29 03:01:14,890 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ad0deeb09f0cf4eec5a424c810de0bf1/info of ad0deeb09f0cf4eec5a424c810de0bf1 into 76b0c38203434b26a3d4f556ae3206f3(size=64.8 K), total size for store is 64.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:01:14,890 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:14,890 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., storeName=ad0deeb09f0cf4eec5a424c810de0bf1/info, priority=13, startTime=1690599674846; duration=0sec 2023-07-29 03:01:14,891 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:16,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:16,828 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:01:16,851 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=167 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/261dfe4b0859480fae11fbca69ee2780 2023-07-29 03:01:16,863 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/261dfe4b0859480fae11fbca69ee2780 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/261dfe4b0859480fae11fbca69ee2780 2023-07-29 03:01:16,866 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-29 03:01:16,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] ipc.CallRunner(144): callId: 165 service: ClientService methodName: Mutate size: 1.2 K connection: 188.40.62.62:55784 deadline: 1690599686866, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:01:16,869 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/261dfe4b0859480fae11fbca69ee2780, entries=8, sequenceid=167, filesize=13.0 K 2023-07-29 03:01:16,870 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for ad0deeb09f0cf4eec5a424c810de0bf1 in 42ms, sequenceid=167, compaction requested=false 2023-07-29 03:01:16,870 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:26,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:26,912 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-07-29 03:01:26,928 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-29 03:01:26,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] ipc.CallRunner(144): callId: 175 service: ClientService methodName: Mutate size: 1.2 K connection: 188.40.62.62:55784 deadline: 1690599696928, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:01:26,935 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=192 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/3f490c2bc5a64fb091e5a858b0e74b1d 2023-07-29 03:01:26,942 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/3f490c2bc5a64fb091e5a858b0e74b1d as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/3f490c2bc5a64fb091e5a858b0e74b1d 2023-07-29 03:01:26,947 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/3f490c2bc5a64fb091e5a858b0e74b1d, entries=22, sequenceid=192, filesize=27.4 K 2023-07-29 03:01:26,948 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for ad0deeb09f0cf4eec5a424c810de0bf1 in 36ms, sequenceid=192, compaction requested=true 2023-07-29 03:01:26,948 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:26,948 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:26,948 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:01:26,949 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 107719 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:01:26,949 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): ad0deeb09f0cf4eec5a424c810de0bf1/info is initiating minor compaction (all files) 2023-07-29 03:01:26,950 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad0deeb09f0cf4eec5a424c810de0bf1/info in TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:26,950 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/76b0c38203434b26a3d4f556ae3206f3, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/261dfe4b0859480fae11fbca69ee2780, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/3f490c2bc5a64fb091e5a858b0e74b1d] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp, totalSize=105.2 K 2023-07-29 03:01:26,950 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 76b0c38203434b26a3d4f556ae3206f3, keycount=58, bloomtype=ROW, size=64.8 K, encoding=NONE, compression=NONE, seqNum=155, earliestPutTs=1690599640535 2023-07-29 03:01:26,951 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 261dfe4b0859480fae11fbca69ee2780, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=167, earliestPutTs=1690599674815 2023-07-29 03:01:26,951 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 3f490c2bc5a64fb091e5a858b0e74b1d, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=192, earliestPutTs=1690599676829 2023-07-29 03:01:26,965 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad0deeb09f0cf4eec5a424c810de0bf1#info#compaction#33 average throughput is 29.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:01:26,980 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/2a8c46b2b7434f798a7dca454c882243 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2a8c46b2b7434f798a7dca454c882243 2023-07-29 03:01:26,988 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ad0deeb09f0cf4eec5a424c810de0bf1/info of ad0deeb09f0cf4eec5a424c810de0bf1 into 2a8c46b2b7434f798a7dca454c882243(size=95.9 K), total size for store is 95.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:01:26,988 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:26,988 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., storeName=ad0deeb09f0cf4eec5a424c810de0bf1/info, priority=13, startTime=1690599686948; duration=0sec 2023-07-29 03:01:26,989 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:36,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:36,952 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:01:36,969 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=204 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/ba1356a1fb1e4842b9db9f89a666d475 2023-07-29 03:01:36,977 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/ba1356a1fb1e4842b9db9f89a666d475 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/ba1356a1fb1e4842b9db9f89a666d475 2023-07-29 03:01:36,982 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/ba1356a1fb1e4842b9db9f89a666d475, entries=8, sequenceid=204, filesize=13.0 K 2023-07-29 03:01:36,982 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=1.03 KB/1051 for ad0deeb09f0cf4eec5a424c810de0bf1 in 30ms, sequenceid=204, compaction requested=false 2023-07-29 03:01:36,982 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:38,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:38,971 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:01:38,986 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=215 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/58367ddbc26e4f988d5687d812c32e44 2023-07-29 03:01:38,992 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/58367ddbc26e4f988d5687d812c32e44 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/58367ddbc26e4f988d5687d812c32e44 2023-07-29 03:01:38,999 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/58367ddbc26e4f988d5687d812c32e44, entries=8, sequenceid=215, filesize=13.0 K 2023-07-29 03:01:39,000 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for ad0deeb09f0cf4eec5a424c810de0bf1 in 29ms, sequenceid=215, compaction requested=true 2023-07-29 03:01:39,000 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:39,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:39,000 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:39,000 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:01:39,000 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-07-29 03:01:39,001 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 124726 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:01:39,002 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): ad0deeb09f0cf4eec5a424c810de0bf1/info is initiating minor compaction (all files) 2023-07-29 03:01:39,002 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad0deeb09f0cf4eec5a424c810de0bf1/info in TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:39,002 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2a8c46b2b7434f798a7dca454c882243, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/ba1356a1fb1e4842b9db9f89a666d475, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/58367ddbc26e4f988d5687d812c32e44] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp, totalSize=121.8 K 2023-07-29 03:01:39,002 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 2a8c46b2b7434f798a7dca454c882243, keycount=88, bloomtype=ROW, size=95.9 K, encoding=NONE, compression=NONE, seqNum=192, earliestPutTs=1690599640535 2023-07-29 03:01:39,003 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting ba1356a1fb1e4842b9db9f89a666d475, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=204, earliestPutTs=1690599686914 2023-07-29 03:01:39,003 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 58367ddbc26e4f988d5687d812c32e44, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=215, earliestPutTs=1690599696954 2023-07-29 03:01:39,017 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=240 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/2577e481c2aa491d932556fa359dc53c 2023-07-29 03:01:39,022 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad0deeb09f0cf4eec5a424c810de0bf1#info#compaction#37 average throughput is 52.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:01:39,024 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/2577e481c2aa491d932556fa359dc53c as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2577e481c2aa491d932556fa359dc53c 2023-07-29 03:01:39,031 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2577e481c2aa491d932556fa359dc53c, entries=22, sequenceid=240, filesize=27.4 K 2023-07-29 03:01:39,032 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=3.08 KB/3153 for ad0deeb09f0cf4eec5a424c810de0bf1 in 32ms, sequenceid=240, compaction requested=false 2023-07-29 03:01:39,032 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:39,034 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/01528ee0696143258c31551951ad62a3 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/01528ee0696143258c31551951ad62a3 2023-07-29 03:01:39,039 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ad0deeb09f0cf4eec5a424c810de0bf1/info of ad0deeb09f0cf4eec5a424c810de0bf1 into 01528ee0696143258c31551951ad62a3(size=112.4 K), total size for store is 139.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:01:39,039 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:39,039 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., storeName=ad0deeb09f0cf4eec5a424c810de0bf1/info, priority=13, startTime=1690599699000; duration=0sec 2023-07-29 03:01:39,040 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:41,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:41,019 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-07-29 03:01:41,029 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=252 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/04ef773d6050464293142d8550c2e87a 2023-07-29 03:01:41,035 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/04ef773d6050464293142d8550c2e87a as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/04ef773d6050464293142d8550c2e87a 2023-07-29 03:01:41,040 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/04ef773d6050464293142d8550c2e87a, entries=8, sequenceid=252, filesize=13.0 K 2023-07-29 03:01:41,041 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=17.45 KB/17867 for ad0deeb09f0cf4eec5a424c810de0bf1 in 22ms, sequenceid=252, compaction requested=true 2023-07-29 03:01:41,041 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:41,041 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:41,041 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:01:41,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:41,042 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=19.50 KB heapSize=21.18 KB 2023-07-29 03:01:41,042 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 156442 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:01:41,043 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): ad0deeb09f0cf4eec5a424c810de0bf1/info is initiating minor compaction (all files) 2023-07-29 03:01:41,043 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad0deeb09f0cf4eec5a424c810de0bf1/info in TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:41,043 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/01528ee0696143258c31551951ad62a3, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2577e481c2aa491d932556fa359dc53c, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/04ef773d6050464293142d8550c2e87a] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp, totalSize=152.8 K 2023-07-29 03:01:41,043 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 01528ee0696143258c31551951ad62a3, keycount=104, bloomtype=ROW, size=112.4 K, encoding=NONE, compression=NONE, seqNum=215, earliestPutTs=1690599640535 2023-07-29 03:01:41,044 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 2577e481c2aa491d932556fa359dc53c, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=240, earliestPutTs=1690599698972 2023-07-29 03:01:41,044 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 04ef773d6050464293142d8550c2e87a, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=252, earliestPutTs=1690599699001 2023-07-29 03:01:41,052 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=19.50 KB at sequenceid=274 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/5d8a1f52f2814500b9182e303a0713ec 2023-07-29 03:01:41,061 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad0deeb09f0cf4eec5a424c810de0bf1#info#compaction#40 average throughput is 33.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:01:41,064 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/5d8a1f52f2814500b9182e303a0713ec as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5d8a1f52f2814500b9182e303a0713ec 2023-07-29 03:01:41,070 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5d8a1f52f2814500b9182e303a0713ec, entries=19, sequenceid=274, filesize=24.3 K 2023-07-29 03:01:41,072 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~19.50 KB/19969, heapSize ~21.16 KB/21672, currentSize=8.21 KB/8408 for ad0deeb09f0cf4eec5a424c810de0bf1 in 30ms, sequenceid=274, compaction requested=false 2023-07-29 03:01:41,072 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:41,077 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/174bb1fec3ea4a31931281e755b6a902 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/174bb1fec3ea4a31931281e755b6a902 2023-07-29 03:01:41,082 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ad0deeb09f0cf4eec5a424c810de0bf1/info of ad0deeb09f0cf4eec5a424c810de0bf1 into 174bb1fec3ea4a31931281e755b6a902(size=143.6 K), total size for store is 167.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:01:41,082 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:41,082 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., storeName=ad0deeb09f0cf4eec5a424c810de0bf1/info, priority=13, startTime=1690599701041; duration=0sec 2023-07-29 03:01:41,082 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:43,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:43,058 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=9.24 KB heapSize=10.16 KB 2023-07-29 03:01:43,075 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.24 KB at sequenceid=287 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/5bd4a45b8bf6479699d6e86a71e9b346 2023-07-29 03:01:43,083 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/5bd4a45b8bf6479699d6e86a71e9b346 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5bd4a45b8bf6479699d6e86a71e9b346 2023-07-29 03:01:43,090 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5bd4a45b8bf6479699d6e86a71e9b346, entries=9, sequenceid=287, filesize=14.0 K 2023-07-29 03:01:43,091 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~9.24 KB/9459, heapSize ~10.15 KB/10392, currentSize=18.47 KB/18918 for ad0deeb09f0cf4eec5a424c810de0bf1 in 33ms, sequenceid=287, compaction requested=true 2023-07-29 03:01:43,091 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:43,091 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:43,091 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:01:43,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:43,092 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=19.50 KB heapSize=21.18 KB 2023-07-29 03:01:43,093 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 186258 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:01:43,093 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): ad0deeb09f0cf4eec5a424c810de0bf1/info is initiating minor compaction (all files) 2023-07-29 03:01:43,093 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad0deeb09f0cf4eec5a424c810de0bf1/info in TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:43,093 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/174bb1fec3ea4a31931281e755b6a902, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5d8a1f52f2814500b9182e303a0713ec, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5bd4a45b8bf6479699d6e86a71e9b346] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp, totalSize=181.9 K 2023-07-29 03:01:43,094 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 174bb1fec3ea4a31931281e755b6a902, keycount=134, bloomtype=ROW, size=143.6 K, encoding=NONE, compression=NONE, seqNum=252, earliestPutTs=1690599640535 2023-07-29 03:01:43,094 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 5d8a1f52f2814500b9182e303a0713ec, keycount=19, bloomtype=ROW, size=24.3 K, encoding=NONE, compression=NONE, seqNum=274, earliestPutTs=1690599701020 2023-07-29 03:01:43,095 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 5bd4a45b8bf6479699d6e86a71e9b346, keycount=9, bloomtype=ROW, size=14.0 K, encoding=NONE, compression=NONE, seqNum=287, earliestPutTs=1690599701043 2023-07-29 03:01:43,108 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-29 03:01:43,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] ipc.CallRunner(144): callId: 273 service: ClientService methodName: Mutate size: 1.2 K connection: 188.40.62.62:55784 deadline: 1690599713107, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ad0deeb09f0cf4eec5a424c810de0bf1, server=jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:01:43,115 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad0deeb09f0cf4eec5a424c810de0bf1#info#compaction#43 average throughput is 81.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:01:43,125 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/9d664d3262724ad0ac250db158755e2f as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/9d664d3262724ad0ac250db158755e2f 2023-07-29 03:01:43,130 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ad0deeb09f0cf4eec5a424c810de0bf1/info of ad0deeb09f0cf4eec5a424c810de0bf1 into 9d664d3262724ad0ac250db158755e2f(size=172.5 K), total size for store is 172.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:01:43,130 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:43,130 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., storeName=ad0deeb09f0cf4eec5a424c810de0bf1/info, priority=13, startTime=1690599703091; duration=0sec 2023-07-29 03:01:43,131 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:43,513 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=19.50 KB at sequenceid=309 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/0f04daebbef0438585485a968ec383da 2023-07-29 03:01:43,555 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/0f04daebbef0438585485a968ec383da as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/0f04daebbef0438585485a968ec383da 2023-07-29 03:01:43,566 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/0f04daebbef0438585485a968ec383da, entries=19, sequenceid=309, filesize=24.3 K 2023-07-29 03:01:43,567 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~19.50 KB/19969, heapSize ~21.16 KB/21672, currentSize=10.26 KB/10510 for ad0deeb09f0cf4eec5a424c810de0bf1 in 475ms, sequenceid=309, compaction requested=false 2023-07-29 03:01:43,567 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:53,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45203] regionserver.HRegion(9158): Flush requested on ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:53,210 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=11.29 KB heapSize=12.37 KB 2023-07-29 03:01:53,222 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=11.29 KB at sequenceid=324 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/be2cb1d6714c475ba299e9ccc55dd3c1 2023-07-29 03:01:53,228 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/be2cb1d6714c475ba299e9ccc55dd3c1 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/be2cb1d6714c475ba299e9ccc55dd3c1 2023-07-29 03:01:53,234 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/be2cb1d6714c475ba299e9ccc55dd3c1, entries=11, sequenceid=324, filesize=16.1 K 2023-07-29 03:01:53,235 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~11.29 KB/11561, heapSize ~12.35 KB/12648, currentSize=1.03 KB/1051 for ad0deeb09f0cf4eec5a424c810de0bf1 in 25ms, sequenceid=324, compaction requested=true 2023-07-29 03:01:53,235 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:53,236 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-29 03:01:53,236 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-29 03:01:53,237 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 217988 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-29 03:01:53,237 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1912): ad0deeb09f0cf4eec5a424c810de0bf1/info is initiating minor compaction (all files) 2023-07-29 03:01:53,237 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad0deeb09f0cf4eec5a424c810de0bf1/info in TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:53,237 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/9d664d3262724ad0ac250db158755e2f, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/0f04daebbef0438585485a968ec383da, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/be2cb1d6714c475ba299e9ccc55dd3c1] into tmpdir=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp, totalSize=212.9 K 2023-07-29 03:01:53,237 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 9d664d3262724ad0ac250db158755e2f, keycount=162, bloomtype=ROW, size=172.5 K, encoding=NONE, compression=NONE, seqNum=287, earliestPutTs=1690599640535 2023-07-29 03:01:53,238 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting 0f04daebbef0438585485a968ec383da, keycount=19, bloomtype=ROW, size=24.3 K, encoding=NONE, compression=NONE, seqNum=309, earliestPutTs=1690599703060 2023-07-29 03:01:53,238 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] compactions.Compactor(207): Compacting be2cb1d6714c475ba299e9ccc55dd3c1, keycount=11, bloomtype=ROW, size=16.1 K, encoding=NONE, compression=NONE, seqNum=324, earliestPutTs=1690599703093 2023-07-29 03:01:53,249 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad0deeb09f0cf4eec5a424c810de0bf1#info#compaction#45 average throughput is 96.22 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-29 03:01:53,259 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/0ad1d8d636674ab88880a4abe6f83d69 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/0ad1d8d636674ab88880a4abe6f83d69 2023-07-29 03:01:53,265 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ad0deeb09f0cf4eec5a424c810de0bf1/info of ad0deeb09f0cf4eec5a424c810de0bf1 into 0ad1d8d636674ab88880a4abe6f83d69(size=203.5 K), total size for store is 203.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-29 03:01:53,265 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:53,265 INFO [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., storeName=ad0deeb09f0cf4eec5a424c810de0bf1/info, priority=13, startTime=1690599713236; duration=0sec 2023-07-29 03:01:53,265 DEBUG [RS:0;jenkins-hbase16:45203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-29 03:01:55,214 INFO [Listener at localhost.localdomain/37169] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-07-29 03:01:55,237 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK] 2023-07-29 03:01:55,237 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK] 2023-07-29 03:01:55,241 INFO [Listener at localhost.localdomain/37169] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 with entries=311, filesize=301.41 KB; new WAL /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599715214 2023-07-29 03:01:55,242 DEBUG [Listener at localhost.localdomain/37169] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK], DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK]] 2023-07-29 03:01:55,242 DEBUG [Listener at localhost.localdomain/37169] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 is not closed yet, will try archiving it next time 2023-07-29 03:01:55,245 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 not finished, retry = 0 2023-07-29 03:01:55,248 INFO [Listener at localhost.localdomain/37169] regionserver.HRegion(2745): Flushing 569434e7c6b63c3e71b650f444f5403b 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-29 03:01:55,259 INFO [Listener at localhost.localdomain/37169] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/.tmp/info/f370b02bcce9418d84f39a0be331fc8b 2023-07-29 03:01:55,265 DEBUG [Listener at localhost.localdomain/37169] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/.tmp/info/f370b02bcce9418d84f39a0be331fc8b as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/info/f370b02bcce9418d84f39a0be331fc8b 2023-07-29 03:01:55,269 INFO [Listener at localhost.localdomain/37169] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/info/f370b02bcce9418d84f39a0be331fc8b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-29 03:01:55,270 INFO [Listener at localhost.localdomain/37169] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 569434e7c6b63c3e71b650f444f5403b in 22ms, sequenceid=6, compaction requested=false 2023-07-29 03:01:55,271 DEBUG [Listener at localhost.localdomain/37169] regionserver.HRegion(2446): Flush status journal for 569434e7c6b63c3e71b650f444f5403b: 2023-07-29 03:01:55,271 INFO [Listener at localhost.localdomain/37169] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-07-29 03:01:55,282 INFO [Listener at localhost.localdomain/37169] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/.tmp/info/bd478d734e114e779c20042a4fca9c5e 2023-07-29 03:01:55,288 DEBUG [Listener at localhost.localdomain/37169] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/.tmp/info/bd478d734e114e779c20042a4fca9c5e as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/info/bd478d734e114e779c20042a4fca9c5e 2023-07-29 03:01:55,292 INFO [Listener at localhost.localdomain/37169] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/info/bd478d734e114e779c20042a4fca9c5e, entries=16, sequenceid=24, filesize=7.0 K 2023-07-29 03:01:55,293 INFO [Listener at localhost.localdomain/37169] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2316, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 22ms, sequenceid=24, compaction requested=false 2023-07-29 03:01:55,293 DEBUG [Listener at localhost.localdomain/37169] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-07-29 03:01:55,293 INFO [Listener at localhost.localdomain/37169] regionserver.HRegion(2745): Flushing ad0deeb09f0cf4eec5a424c810de0bf1 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-07-29 03:01:55,304 INFO [Listener at localhost.localdomain/37169] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=329 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/b7611ad31f044f7e9e02e9572e6c2c55 2023-07-29 03:01:55,311 DEBUG [Listener at localhost.localdomain/37169] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/.tmp/info/b7611ad31f044f7e9e02e9572e6c2c55 as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/b7611ad31f044f7e9e02e9572e6c2c55 2023-07-29 03:01:55,317 INFO [Listener at localhost.localdomain/37169] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/b7611ad31f044f7e9e02e9572e6c2c55, entries=1, sequenceid=329, filesize=5.8 K 2023-07-29 03:01:55,319 INFO [Listener at localhost.localdomain/37169] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for ad0deeb09f0cf4eec5a424c810de0bf1 in 25ms, sequenceid=329, compaction requested=false 2023-07-29 03:01:55,319 DEBUG [Listener at localhost.localdomain/37169] regionserver.HRegion(2446): Flush status journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:55,319 DEBUG [Listener at localhost.localdomain/37169] regionserver.HRegion(2446): Flush status journal for 1335e692f5ac52c1fcc86a408e0ec39c: 2023-07-29 03:01:55,333 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK] 2023-07-29 03:01:55,334 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK] 2023-07-29 03:01:55,335 INFO [Listener at localhost.localdomain/37169] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599715214 with entries=4, filesize=1.22 KB; new WAL /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599715319 2023-07-29 03:01:55,336 DEBUG [Listener at localhost.localdomain/37169] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43217,DS-e97e468f-b572-452c-96c7-167f05d1941f,DISK], DatanodeInfoWithStorage[127.0.0.1:40221,DS-5b4b3f87-948f-49b3-960e-6588d7f616bd,DISK]] 2023-07-29 03:01:55,336 DEBUG [Listener at localhost.localdomain/37169] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 is not closed yet, will try archiving it next time 2023-07-29 03:01:55,336 DEBUG [Listener at localhost.localdomain/37169] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599715214 is not closed yet, will try archiving it next time 2023-07-29 03:01:55,337 INFO [Listener at localhost.localdomain/37169] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-07-29 03:01:55,338 DEBUG [Close-WAL-Writer-1] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 is not closed yet, will try archiving it next time 2023-07-29 03:01:55,339 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599715214 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/oldWALs/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599715214 2023-07-29 03:01:55,346 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/oldWALs/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599627430 2023-07-29 03:01:55,437 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-29 03:01:55,437 INFO [Listener at localhost.localdomain/37169] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-29 03:01:55,437 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6415412a to 127.0.0.1:57405 2023-07-29 03:01:55,437 DEBUG [Listener at localhost.localdomain/37169] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:55,438 DEBUG [Listener at localhost.localdomain/37169] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-29 03:01:55,438 DEBUG [Listener at localhost.localdomain/37169] util.JVMClusterUtil(257): Found active master hash=610095237, stopped=false 2023-07-29 03:01:55,438 INFO [Listener at localhost.localdomain/37169] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:01:55,448 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 03:01:55,448 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 03:01:55,448 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:55,448 INFO [Listener at localhost.localdomain/37169] procedure2.ProcedureExecutor(629): Stopping 2023-07-29 03:01:55,449 DEBUG [Listener at localhost.localdomain/37169] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7eb7d79d to 127.0.0.1:57405 2023-07-29 03:01:55,450 DEBUG [Listener at localhost.localdomain/37169] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:55,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:01:55,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:01:55,450 INFO [Listener at localhost.localdomain/37169] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,45203,1690599626908' ***** 2023-07-29 03:01:55,451 INFO [Listener at localhost.localdomain/37169] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-29 03:01:55,451 INFO [RS:0;jenkins-hbase16:45203] regionserver.HeapMemoryManager(220): Stopping 2023-07-29 03:01:55,451 INFO [RS:0;jenkins-hbase16:45203] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-29 03:01:55,451 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-29 03:01:55,451 INFO [RS:0;jenkins-hbase16:45203] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-29 03:01:55,452 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(3305): Received CLOSE for 569434e7c6b63c3e71b650f444f5403b 2023-07-29 03:01:55,452 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(3305): Received CLOSE for ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:55,452 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(3305): Received CLOSE for 1335e692f5ac52c1fcc86a408e0ec39c 2023-07-29 03:01:55,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 569434e7c6b63c3e71b650f444f5403b, disabling compactions & flushes 2023-07-29 03:01:55,452 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:01:55,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:01:55,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:01:55,452 DEBUG [RS:0;jenkins-hbase16:45203] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x42b142dc to 127.0.0.1:57405 2023-07-29 03:01:55,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. after waiting 0 ms 2023-07-29 03:01:55,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:01:55,453 DEBUG [RS:0;jenkins-hbase16:45203] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:55,453 INFO [RS:0;jenkins-hbase16:45203] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-29 03:01:55,453 INFO [RS:0;jenkins-hbase16:45203] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-29 03:01:55,453 INFO [RS:0;jenkins-hbase16:45203] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-29 03:01:55,453 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-29 03:01:55,454 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-29 03:01:55,456 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 03:01:55,456 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1478): Online Regions={569434e7c6b63c3e71b650f444f5403b=hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b., 1588230740=hbase:meta,,1.1588230740, ad0deeb09f0cf4eec5a424c810de0bf1=TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1., 1335e692f5ac52c1fcc86a408e0ec39c=TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.} 2023-07-29 03:01:55,456 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 03:01:55,457 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 03:01:55,457 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 03:01:55,457 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 03:01:55,457 DEBUG [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1504): Waiting on 1335e692f5ac52c1fcc86a408e0ec39c, 1588230740, 569434e7c6b63c3e71b650f444f5403b, ad0deeb09f0cf4eec5a424c810de0bf1 2023-07-29 03:01:55,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/namespace/569434e7c6b63c3e71b650f444f5403b/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-29 03:01:55,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 569434e7c6b63c3e71b650f444f5403b: 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690599627711.569434e7c6b63c3e71b650f444f5403b. 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing ad0deeb09f0cf4eec5a424c810de0bf1, disabling compactions & flushes 2023-07-29 03:01:55,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. after waiting 0 ms 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:55,465 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 03:01:55,466 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 03:01:55,467 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 03:01:55,469 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-29 03:01:55,480 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0->hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a-top, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-583d9e68e9bf47b98e881ca39c456150, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/e51dda86136a445e8ffbf3cb09b1d591, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-14c40be58689445ea139a37e6985c220, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/42269e43617c414fb82459adaa508461, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/76b0c38203434b26a3d4f556ae3206f3, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/035ee277d75f4e6e9c226ebeb40b2a9e, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/261dfe4b0859480fae11fbca69ee2780, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2a8c46b2b7434f798a7dca454c882243, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/3f490c2bc5a64fb091e5a858b0e74b1d, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/ba1356a1fb1e4842b9db9f89a666d475, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/01528ee0696143258c31551951ad62a3, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/58367ddbc26e4f988d5687d812c32e44, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2577e481c2aa491d932556fa359dc53c, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/174bb1fec3ea4a31931281e755b6a902, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/04ef773d6050464293142d8550c2e87a, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5d8a1f52f2814500b9182e303a0713ec, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/9d664d3262724ad0ac250db158755e2f, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5bd4a45b8bf6479699d6e86a71e9b346, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/0f04daebbef0438585485a968ec383da, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/be2cb1d6714c475ba299e9ccc55dd3c1] to archive 2023-07-29 03:01:55,481 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-29 03:01:55,483 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:01:55,484 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-583d9e68e9bf47b98e881ca39c456150 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-583d9e68e9bf47b98e881ca39c456150 2023-07-29 03:01:55,486 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/e51dda86136a445e8ffbf3cb09b1d591 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/e51dda86136a445e8ffbf3cb09b1d591 2023-07-29 03:01:55,487 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-14c40be58689445ea139a37e6985c220 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/TestLogRolling-testLogRolling=6edf31243051b9aa72f064147bfc47d0-14c40be58689445ea139a37e6985c220 2023-07-29 03:01:55,488 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/42269e43617c414fb82459adaa508461 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/42269e43617c414fb82459adaa508461 2023-07-29 03:01:55,490 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/76b0c38203434b26a3d4f556ae3206f3 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/76b0c38203434b26a3d4f556ae3206f3 2023-07-29 03:01:55,491 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/035ee277d75f4e6e9c226ebeb40b2a9e to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/035ee277d75f4e6e9c226ebeb40b2a9e 2023-07-29 03:01:55,493 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/261dfe4b0859480fae11fbca69ee2780 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/261dfe4b0859480fae11fbca69ee2780 2023-07-29 03:01:55,494 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2a8c46b2b7434f798a7dca454c882243 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2a8c46b2b7434f798a7dca454c882243 2023-07-29 03:01:55,496 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/3f490c2bc5a64fb091e5a858b0e74b1d to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/3f490c2bc5a64fb091e5a858b0e74b1d 2023-07-29 03:01:55,497 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/ba1356a1fb1e4842b9db9f89a666d475 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/ba1356a1fb1e4842b9db9f89a666d475 2023-07-29 03:01:55,499 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/01528ee0696143258c31551951ad62a3 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/01528ee0696143258c31551951ad62a3 2023-07-29 03:01:55,500 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/58367ddbc26e4f988d5687d812c32e44 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/58367ddbc26e4f988d5687d812c32e44 2023-07-29 03:01:55,502 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2577e481c2aa491d932556fa359dc53c to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/2577e481c2aa491d932556fa359dc53c 2023-07-29 03:01:55,504 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/174bb1fec3ea4a31931281e755b6a902 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/174bb1fec3ea4a31931281e755b6a902 2023-07-29 03:01:55,506 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/04ef773d6050464293142d8550c2e87a to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/04ef773d6050464293142d8550c2e87a 2023-07-29 03:01:55,508 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5d8a1f52f2814500b9182e303a0713ec to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5d8a1f52f2814500b9182e303a0713ec 2023-07-29 03:01:55,509 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/9d664d3262724ad0ac250db158755e2f to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/9d664d3262724ad0ac250db158755e2f 2023-07-29 03:01:55,510 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5bd4a45b8bf6479699d6e86a71e9b346 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/5bd4a45b8bf6479699d6e86a71e9b346 2023-07-29 03:01:55,511 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/0f04daebbef0438585485a968ec383da to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/0f04daebbef0438585485a968ec383da 2023-07-29 03:01:55,512 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/be2cb1d6714c475ba299e9ccc55dd3c1 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/info/be2cb1d6714c475ba299e9ccc55dd3c1 2023-07-29 03:01:55,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/ad0deeb09f0cf4eec5a424c810de0bf1/recovered.edits/332.seqid, newMaxSeqId=332, maxSeqId=121 2023-07-29 03:01:55,517 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:55,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for ad0deeb09f0cf4eec5a424c810de0bf1: 2023-07-29 03:01:55,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1690599642634.ad0deeb09f0cf4eec5a424c810de0bf1. 2023-07-29 03:01:55,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1335e692f5ac52c1fcc86a408e0ec39c, disabling compactions & flushes 2023-07-29 03:01:55,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:01:55,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:01:55,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. after waiting 0 ms 2023-07-29 03:01:55,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:01:55,518 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0->hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/6edf31243051b9aa72f064147bfc47d0/info/7b5d7d6008514f1b856081458363714a-bottom] to archive 2023-07-29 03:01:55,519 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-29 03:01:55,520 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0 to hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/archive/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/info/7b5d7d6008514f1b856081458363714a.6edf31243051b9aa72f064147bfc47d0 2023-07-29 03:01:55,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/data/default/TestLogRolling-testLogRolling/1335e692f5ac52c1fcc86a408e0ec39c/recovered.edits/126.seqid, newMaxSeqId=126, maxSeqId=121 2023-07-29 03:01:55,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:01:55,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1335e692f5ac52c1fcc86a408e0ec39c: 2023-07-29 03:01:55,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1690599642634.1335e692f5ac52c1fcc86a408e0ec39c. 2023-07-29 03:01:55,657 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,45203,1690599626908; all regions closed. 2023-07-29 03:01:55,668 DEBUG [RS:0;jenkins-hbase16:45203] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/oldWALs 2023-07-29 03:01:55,668 INFO [RS:0;jenkins-hbase16:45203] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C45203%2C1690599626908.meta:.meta(num 1690599627629) 2023-07-29 03:01:55,671 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/WALs/jenkins-hbase16.apache.org,45203,1690599626908/jenkins-hbase16.apache.org%2C45203%2C1690599626908.1690599715319 not finished, retry = 0 2023-07-29 03:01:55,777 DEBUG [RS:0;jenkins-hbase16:45203] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/oldWALs 2023-07-29 03:01:55,777 INFO [RS:0;jenkins-hbase16:45203] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C45203%2C1690599626908:(num 1690599715319) 2023-07-29 03:01:55,777 DEBUG [RS:0;jenkins-hbase16:45203] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:55,777 INFO [RS:0;jenkins-hbase16:45203] regionserver.LeaseManager(133): Closed leases 2023-07-29 03:01:55,778 INFO [RS:0;jenkins-hbase16:45203] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-29 03:01:55,778 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 03:01:55,780 INFO [RS:0;jenkins-hbase16:45203] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:45203 2023-07-29 03:01:55,815 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,45203,1690599626908 2023-07-29 03:01:55,815 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:01:55,815 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:01:55,828 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,45203,1690599626908] 2023-07-29 03:01:55,829 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,45203,1690599626908; numProcessing=1 2023-07-29 03:01:55,840 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,45203,1690599626908 already deleted, retry=false 2023-07-29 03:01:55,840 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,45203,1690599626908 expired; onlineServers=0 2023-07-29 03:01:55,840 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,38955,1690599626768' ***** 2023-07-29 03:01:55,840 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-29 03:01:55,842 DEBUG [M:0;jenkins-hbase16:38955] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2062d0d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 03:01:55,842 INFO [M:0;jenkins-hbase16:38955] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,38955,1690599626768 2023-07-29 03:01:55,842 INFO [M:0;jenkins-hbase16:38955] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,38955,1690599626768; all regions closed. 2023-07-29 03:01:55,842 DEBUG [M:0;jenkins-hbase16:38955] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:55,843 DEBUG [M:0;jenkins-hbase16:38955] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-29 03:01:55,843 DEBUG [M:0;jenkins-hbase16:38955] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-29 03:01:55,843 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599627229] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599627229,5,FailOnTimeoutGroup] 2023-07-29 03:01:55,844 INFO [M:0;jenkins-hbase16:38955] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-29 03:01:55,843 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-29 03:01:55,843 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599627230] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599627230,5,FailOnTimeoutGroup] 2023-07-29 03:01:55,846 INFO [M:0;jenkins-hbase16:38955] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-29 03:01:55,847 INFO [M:0;jenkins-hbase16:38955] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-07-29 03:01:55,847 DEBUG [M:0;jenkins-hbase16:38955] master.HMaster(1512): Stopping service threads 2023-07-29 03:01:55,848 INFO [M:0;jenkins-hbase16:38955] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-29 03:01:55,848 ERROR [M:0;jenkins-hbase16:38955] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-29 03:01:55,849 INFO [M:0;jenkins-hbase16:38955] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-29 03:01:55,849 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-29 03:01:55,853 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-29 03:01:55,853 DEBUG [M:0;jenkins-hbase16:38955] zookeeper.ZKUtil(398): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-29 03:01:55,853 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:55,853 WARN [M:0;jenkins-hbase16:38955] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-29 03:01:55,853 INFO [M:0;jenkins-hbase16:38955] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-29 03:01:55,854 INFO [M:0;jenkins-hbase16:38955] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-29 03:01:55,855 DEBUG [M:0;jenkins-hbase16:38955] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 03:01:55,855 INFO [M:0;jenkins-hbase16:38955] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:55,855 DEBUG [M:0;jenkins-hbase16:38955] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:55,855 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 03:01:55,855 DEBUG [M:0;jenkins-hbase16:38955] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 03:01:55,855 DEBUG [M:0;jenkins-hbase16:38955] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:55,855 INFO [M:0;jenkins-hbase16:38955] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.78 KB heapSize=78.52 KB 2023-07-29 03:01:55,873 INFO [M:0;jenkins-hbase16:38955] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.78 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/90e66231c506464f88e526fbdc2c95dc 2023-07-29 03:01:55,878 INFO [M:0;jenkins-hbase16:38955] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90e66231c506464f88e526fbdc2c95dc 2023-07-29 03:01:55,879 DEBUG [M:0;jenkins-hbase16:38955] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/90e66231c506464f88e526fbdc2c95dc as hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/90e66231c506464f88e526fbdc2c95dc 2023-07-29 03:01:55,883 INFO [M:0;jenkins-hbase16:38955] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90e66231c506464f88e526fbdc2c95dc 2023-07-29 03:01:55,883 INFO [M:0;jenkins-hbase16:38955] regionserver.HStore(1080): Added hdfs://localhost.localdomain:41579/user/jenkins/test-data/841f1343-9bf3-0afc-f24e-70e2923ba0e3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/90e66231c506464f88e526fbdc2c95dc, entries=18, sequenceid=160, filesize=6.9 K 2023-07-29 03:01:55,884 INFO [M:0;jenkins-hbase16:38955] regionserver.HRegion(2948): Finished flush of dataSize ~64.78 KB/66332, heapSize ~78.51 KB/80392, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=160, compaction requested=false 2023-07-29 03:01:55,885 INFO [M:0;jenkins-hbase16:38955] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:55,885 DEBUG [M:0;jenkins-hbase16:38955] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 03:01:55,888 INFO [M:0;jenkins-hbase16:38955] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-29 03:01:55,888 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 03:01:55,888 INFO [M:0;jenkins-hbase16:38955] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:38955 2023-07-29 03:01:55,898 DEBUG [M:0;jenkins-hbase16:38955] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,38955,1690599626768 already deleted, retry=false 2023-07-29 03:01:55,929 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:01:55,929 INFO [RS:0;jenkins-hbase16:45203] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,45203,1690599626908; zookeeper connection closed. 2023-07-29 03:01:55,929 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): regionserver:45203-0x101af0556410001, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:01:55,930 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@35c56084] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@35c56084 2023-07-29 03:01:55,930 INFO [Listener at localhost.localdomain/37169] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-29 03:01:56,029 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:01:56,029 INFO [M:0;jenkins-hbase16:38955] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,38955,1690599626768; zookeeper connection closed. 2023-07-29 03:01:56,029 DEBUG [Listener at localhost.localdomain/37169-EventThread] zookeeper.ZKWatcher(600): master:38955-0x101af0556410000, quorum=127.0.0.1:57405, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:01:56,031 WARN [Listener at localhost.localdomain/37169] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 03:01:56,041 INFO [Listener at localhost.localdomain/37169] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 03:01:56,151 WARN [BP-91098139-188.40.62.62-1690599625360 heartbeating to localhost.localdomain/127.0.0.1:41579] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 03:01:56,151 WARN [BP-91098139-188.40.62.62-1690599625360 heartbeating to localhost.localdomain/127.0.0.1:41579] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-91098139-188.40.62.62-1690599625360 (Datanode Uuid 0ecd9ce8-c6f6-4c5b-b972-168c0f2addc8) service to localhost.localdomain/127.0.0.1:41579 2023-07-29 03:01:56,153 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13/dfs/data/data3/current/BP-91098139-188.40.62.62-1690599625360] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:01:56,154 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13/dfs/data/data4/current/BP-91098139-188.40.62.62-1690599625360] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:01:56,155 WARN [Listener at localhost.localdomain/37169] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 03:01:56,161 INFO [Listener at localhost.localdomain/37169] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 03:01:56,265 WARN [BP-91098139-188.40.62.62-1690599625360 heartbeating to localhost.localdomain/127.0.0.1:41579] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 03:01:56,265 WARN [BP-91098139-188.40.62.62-1690599625360 heartbeating to localhost.localdomain/127.0.0.1:41579] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-91098139-188.40.62.62-1690599625360 (Datanode Uuid 8fadec19-dd7b-4c7c-befb-4c17a3872ee7) service to localhost.localdomain/127.0.0.1:41579 2023-07-29 03:01:56,266 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13/dfs/data/data1/current/BP-91098139-188.40.62.62-1690599625360] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:01:56,267 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/cluster_2daa35bf-2bcd-66ce-e2af-7a83af6eea13/dfs/data/data2/current/BP-91098139-188.40.62.62-1690599625360] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:01:56,279 INFO [Listener at localhost.localdomain/37169] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-07-29 03:01:56,400 INFO [Listener at localhost.localdomain/37169] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-29 03:01:56,432 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-29 03:01:56,440 INFO [Listener at localhost.localdomain/37169] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=76 (was 66) - Thread LEAK? -, OpenFileDescriptor=467 (was 435) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=36 (was 146), ProcessCount=165 (was 169), AvailableMemoryMB=5613 (was 4525) - AvailableMemoryMB LEAK? - 2023-07-29 03:01:56,448 INFO [Listener at localhost.localdomain/37169] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=77, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=36, ProcessCount=165, AvailableMemoryMB=5612 2023-07-29 03:01:56,448 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-29 03:01:56,448 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/hadoop.log.dir so I do NOT create it in target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1 2023-07-29 03:01:56,448 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8615808d-2518-1170-fcc7-70582fd5d78f/hadoop.tmp.dir so I do NOT create it in target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1 2023-07-29 03:01:56,448 INFO [Listener at localhost.localdomain/37169] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1, deleteOnExit=true 2023-07-29 03:01:56,448 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-29 03:01:56,448 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/test.cache.data in system properties and HBase conf 2023-07-29 03:01:56,449 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/hadoop.tmp.dir in system properties and HBase conf 2023-07-29 03:01:56,449 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/hadoop.log.dir in system properties and HBase conf 2023-07-29 03:01:56,449 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-29 03:01:56,449 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-29 03:01:56,449 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-29 03:01:56,449 DEBUG [Listener at localhost.localdomain/37169] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-29 03:01:56,449 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-29 03:01:56,449 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-29 03:01:56,450 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/nfs.dump.dir in system properties and HBase conf 2023-07-29 03:01:56,451 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/java.io.tmpdir in system properties and HBase conf 2023-07-29 03:01:56,451 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-29 03:01:56,451 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-29 03:01:56,451 INFO [Listener at localhost.localdomain/37169] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-29 03:01:56,453 WARN [Listener at localhost.localdomain/37169] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 03:01:56,453 WARN [Listener at localhost.localdomain/37169] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 03:01:56,670 WARN [Listener at localhost.localdomain/37169] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 03:01:56,673 INFO [Listener at localhost.localdomain/37169] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 03:01:56,679 INFO [Listener at localhost.localdomain/37169] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/java.io.tmpdir/Jetty_localhost_localdomain_45569_hdfs____.p4uh73/webapp 2023-07-29 03:01:56,750 INFO [Listener at localhost.localdomain/37169] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:45569 2023-07-29 03:01:56,752 WARN [Listener at localhost.localdomain/37169] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-29 03:01:56,753 WARN [Listener at localhost.localdomain/37169] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-29 03:01:56,879 WARN [Listener at localhost.localdomain/44797] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 03:01:56,890 WARN [Listener at localhost.localdomain/44797] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 03:01:56,892 WARN [Listener at localhost.localdomain/44797] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 03:01:56,894 INFO [Listener at localhost.localdomain/44797] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 03:01:56,901 INFO [Listener at localhost.localdomain/44797] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/java.io.tmpdir/Jetty_localhost_42509_datanode____67me93/webapp 2023-07-29 03:01:56,973 INFO [Listener at localhost.localdomain/44797] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42509 2023-07-29 03:01:56,978 WARN [Listener at localhost.localdomain/39009] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 03:01:56,988 WARN [Listener at localhost.localdomain/39009] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-29 03:01:56,990 WARN [Listener at localhost.localdomain/39009] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-29 03:01:56,992 INFO [Listener at localhost.localdomain/39009] log.Slf4jLog(67): jetty-6.1.26 2023-07-29 03:01:56,998 INFO [Listener at localhost.localdomain/39009] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/java.io.tmpdir/Jetty_localhost_46595_datanode____.cdbxhm/webapp 2023-07-29 03:01:57,073 INFO [Listener at localhost.localdomain/39009] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46595 2023-07-29 03:01:57,080 WARN [Listener at localhost.localdomain/46205] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-29 03:01:57,312 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-29 03:01:57,523 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xabff420229ba99b9: Processing first storage report for DS-01c0b669-5793-408a-a5b4-1a2ccea88731 from datanode 04e4a617-7874-427d-a4dc-3f6dac385920 2023-07-29 03:01:57,523 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xabff420229ba99b9: from storage DS-01c0b669-5793-408a-a5b4-1a2ccea88731 node DatanodeRegistration(127.0.0.1:32795, datanodeUuid=04e4a617-7874-427d-a4dc-3f6dac385920, infoPort=46109, infoSecurePort=0, ipcPort=39009, storageInfo=lv=-57;cid=testClusterID;nsid=135514453;c=1690599716455), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:01:57,523 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xabff420229ba99b9: Processing first storage report for DS-4b29bc07-45c7-4577-99d3-1cfc3644b179 from datanode 04e4a617-7874-427d-a4dc-3f6dac385920 2023-07-29 03:01:57,523 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xabff420229ba99b9: from storage DS-4b29bc07-45c7-4577-99d3-1cfc3644b179 node DatanodeRegistration(127.0.0.1:32795, datanodeUuid=04e4a617-7874-427d-a4dc-3f6dac385920, infoPort=46109, infoSecurePort=0, ipcPort=39009, storageInfo=lv=-57;cid=testClusterID;nsid=135514453;c=1690599716455), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:01:57,700 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9bc06398457ea18b: Processing first storage report for DS-9ce1620b-7955-4a95-8632-73c3d2eddf01 from datanode 376964fd-866a-4a1f-8b7e-da1ab06ea1d1 2023-07-29 03:01:57,700 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9bc06398457ea18b: from storage DS-9ce1620b-7955-4a95-8632-73c3d2eddf01 node DatanodeRegistration(127.0.0.1:45725, datanodeUuid=376964fd-866a-4a1f-8b7e-da1ab06ea1d1, infoPort=39035, infoSecurePort=0, ipcPort=46205, storageInfo=lv=-57;cid=testClusterID;nsid=135514453;c=1690599716455), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:01:57,700 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9bc06398457ea18b: Processing first storage report for DS-c5f4539d-0ddd-4012-bdb9-c7812d2b2d15 from datanode 376964fd-866a-4a1f-8b7e-da1ab06ea1d1 2023-07-29 03:01:57,700 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9bc06398457ea18b: from storage DS-c5f4539d-0ddd-4012-bdb9-c7812d2b2d15 node DatanodeRegistration(127.0.0.1:45725, datanodeUuid=376964fd-866a-4a1f-8b7e-da1ab06ea1d1, infoPort=39035, infoSecurePort=0, ipcPort=46205, storageInfo=lv=-57;cid=testClusterID;nsid=135514453;c=1690599716455), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-29 03:01:57,801 DEBUG [Listener at localhost.localdomain/46205] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1 2023-07-29 03:01:57,806 INFO [Listener at localhost.localdomain/46205] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1/zookeeper_0, clientPort=64034, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-29 03:01:57,808 INFO [Listener at localhost.localdomain/46205] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64034 2023-07-29 03:01:57,809 INFO [Listener at localhost.localdomain/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:57,810 INFO [Listener at localhost.localdomain/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:57,825 INFO [Listener at localhost.localdomain/46205] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092 with version=8 2023-07-29 03:01:57,825 INFO [Listener at localhost.localdomain/46205] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost.localdomain:45449/user/jenkins/test-data/ee0c865e-3b29-07d2-39fc-c3e4ec97f55c/hbase-staging 2023-07-29 03:01:57,827 INFO [Listener at localhost.localdomain/46205] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 03:01:57,828 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:01:57,828 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 03:01:57,828 INFO [Listener at localhost.localdomain/46205] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 03:01:57,828 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:01:57,828 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 03:01:57,828 INFO [Listener at localhost.localdomain/46205] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 03:01:57,830 INFO [Listener at localhost.localdomain/46205] ipc.NettyRpcServer(120): Bind to /188.40.62.62:41697 2023-07-29 03:01:57,831 INFO [Listener at localhost.localdomain/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:57,832 INFO [Listener at localhost.localdomain/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:57,833 INFO [Listener at localhost.localdomain/46205] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41697 connecting to ZooKeeper ensemble=127.0.0.1:64034 2023-07-29 03:01:57,875 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:416970x0, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 03:01:57,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41697-0x101af06b9f10000 connected 2023-07-29 03:01:57,938 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 03:01:57,939 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:01:57,940 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 03:01:57,941 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41697 2023-07-29 03:01:57,941 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41697 2023-07-29 03:01:57,942 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41697 2023-07-29 03:01:57,942 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41697 2023-07-29 03:01:57,942 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41697 2023-07-29 03:01:57,943 INFO [Listener at localhost.localdomain/46205] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092, hbase.cluster.distributed=false 2023-07-29 03:01:57,959 INFO [Listener at localhost.localdomain/46205] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-07-29 03:01:57,959 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:01:57,959 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-29 03:01:57,959 INFO [Listener at localhost.localdomain/46205] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-29 03:01:57,959 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-29 03:01:57,959 INFO [Listener at localhost.localdomain/46205] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-29 03:01:57,959 INFO [Listener at localhost.localdomain/46205] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-29 03:01:57,962 INFO [Listener at localhost.localdomain/46205] ipc.NettyRpcServer(120): Bind to /188.40.62.62:44149 2023-07-29 03:01:57,962 INFO [Listener at localhost.localdomain/46205] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-29 03:01:57,963 DEBUG [Listener at localhost.localdomain/46205] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-29 03:01:57,963 INFO [Listener at localhost.localdomain/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:57,964 INFO [Listener at localhost.localdomain/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:57,965 INFO [Listener at localhost.localdomain/46205] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44149 connecting to ZooKeeper ensemble=127.0.0.1:64034 2023-07-29 03:01:57,971 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:441490x0, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-29 03:01:57,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44149-0x101af06b9f10001 connected 2023-07-29 03:01:57,973 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ZKUtil(164): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 03:01:57,973 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ZKUtil(164): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:01:57,974 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ZKUtil(164): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-29 03:01:57,975 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44149 2023-07-29 03:01:57,975 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44149 2023-07-29 03:01:57,975 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44149 2023-07-29 03:01:57,975 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44149 2023-07-29 03:01:57,975 DEBUG [Listener at localhost.localdomain/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44149 2023-07-29 03:01:57,976 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:57,986 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 03:01:57,987 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:57,996 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 03:01:57,996 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-29 03:01:57,996 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:57,997 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 03:01:57,999 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,41697,1690599717827 from backup master directory 2023-07-29 03:01:57,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-29 03:01:58,004 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:58,005 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 03:01:58,005 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-29 03:01:58,005 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:58,020 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/hbase.id with ID: 24374fef-f8e9-4941-9a86-d1a6fa139075 2023-07-29 03:01:58,029 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:58,038 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:58,047 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7590d15f to 127.0.0.1:64034 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 03:01:58,056 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49ab7d39, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 03:01:58,056 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-29 03:01:58,057 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-29 03:01:58,057 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 03:01:58,059 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store-tmp 2023-07-29 03:01:58,070 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:01:58,070 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 03:01:58,071 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:58,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:58,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 03:01:58,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:58,071 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:58,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 03:01:58,071 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/WALs/jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:58,075 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C41697%2C1690599717827, suffix=, logDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/WALs/jenkins-hbase16.apache.org,41697,1690599717827, archiveDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/oldWALs, maxLogs=10 2023-07-29 03:01:58,088 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK] 2023-07-29 03:01:58,088 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK] 2023-07-29 03:01:58,090 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/WALs/jenkins-hbase16.apache.org,41697,1690599717827/jenkins-hbase16.apache.org%2C41697%2C1690599717827.1690599718075 2023-07-29 03:01:58,090 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK], DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK]] 2023-07-29 03:01:58,090 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-29 03:01:58,091 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:01:58,091 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:01:58,091 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:01:58,093 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:01:58,094 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-29 03:01:58,094 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-29 03:01:58,095 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:58,096 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:01:58,096 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:01:58,098 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-29 03:01:58,100 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 03:01:58,101 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=767893, jitterRate=-0.023574724793434143}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 03:01:58,101 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 03:01:58,101 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-29 03:01:58,102 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-29 03:01:58,102 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-29 03:01:58,102 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-29 03:01:58,102 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-29 03:01:58,103 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-29 03:01:58,103 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-29 03:01:58,103 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-29 03:01:58,104 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-29 03:01:58,114 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-29 03:01:58,115 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-29 03:01:58,115 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-29 03:01:58,115 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-29 03:01:58,116 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-29 03:01:58,121 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:58,121 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-29 03:01:58,122 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-29 03:01:58,123 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-29 03:01:58,129 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 03:01:58,129 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:58,129 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-29 03:01:58,130 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase16.apache.org,41697,1690599717827, sessionid=0x101af06b9f10000, setting cluster-up flag (Was=false) 2023-07-29 03:01:58,146 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:58,171 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-29 03:01:58,173 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:58,188 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:58,213 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-29 03:01:58,215 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:58,217 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/.hbase-snapshot/.tmp 2023-07-29 03:01:58,220 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-29 03:01:58,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:01:58,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:01:58,221 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:01:58,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-07-29 03:01:58,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-07-29 03:01:58,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 03:01:58,222 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,224 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690599748224 2023-07-29 03:01:58,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-29 03:01:58,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-29 03:01:58,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-29 03:01:58,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-29 03:01:58,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-29 03:01:58,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-29 03:01:58,225 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 03:01:58,225 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,225 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-29 03:01:58,226 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-29 03:01:58,226 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-29 03:01:58,227 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-29 03:01:58,227 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-29 03:01:58,227 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-29 03:01:58,227 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599718227,5,FailOnTimeoutGroup] 2023-07-29 03:01:58,228 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 03:01:58,228 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599718227,5,FailOnTimeoutGroup] 2023-07-29 03:01:58,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-29 03:01:58,228 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,229 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,239 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 03:01:58,239 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-29 03:01:58,239 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092 2023-07-29 03:01:58,247 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:01:58,248 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 03:01:58,249 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/info 2023-07-29 03:01:58,249 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 03:01:58,250 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:58,250 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 03:01:58,251 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/rep_barrier 2023-07-29 03:01:58,251 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 03:01:58,252 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:58,252 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 03:01:58,253 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/table 2023-07-29 03:01:58,253 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 03:01:58,254 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:58,255 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740 2023-07-29 03:01:58,255 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740 2023-07-29 03:01:58,257 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 03:01:58,259 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 03:01:58,261 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 03:01:58,261 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=718451, jitterRate=-0.08644348382949829}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 03:01:58,261 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 03:01:58,262 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 03:01:58,262 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 03:01:58,262 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 03:01:58,262 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 03:01:58,262 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 03:01:58,262 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 03:01:58,262 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 03:01:58,263 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-29 03:01:58,263 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-29 03:01:58,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-29 03:01:58,265 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-29 03:01:58,266 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-29 03:01:58,278 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(951): ClusterId : 24374fef-f8e9-4941-9a86-d1a6fa139075 2023-07-29 03:01:58,279 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-29 03:01:58,290 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-29 03:01:58,290 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-29 03:01:58,298 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-29 03:01:58,300 DEBUG [RS:0;jenkins-hbase16:44149] zookeeper.ReadOnlyZKClient(139): Connect 0x2a08acb4 to 127.0.0.1:64034 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 03:01:58,315 DEBUG [RS:0;jenkins-hbase16:44149] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24cf5785, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 03:01:58,315 DEBUG [RS:0;jenkins-hbase16:44149] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20f0c7c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 03:01:58,330 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:44149 2023-07-29 03:01:58,330 INFO [RS:0;jenkins-hbase16:44149] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-29 03:01:58,330 INFO [RS:0;jenkins-hbase16:44149] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-29 03:01:58,330 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1022): About to register with Master. 2023-07-29 03:01:58,331 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase16.apache.org,41697,1690599717827 with isa=jenkins-hbase16.apache.org/188.40.62.62:44149, startcode=1690599717958 2023-07-29 03:01:58,331 DEBUG [RS:0;jenkins-hbase16:44149] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-29 03:01:58,333 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:53263, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-29 03:01:58,334 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41697] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,334 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092 2023-07-29 03:01:58,334 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:44797 2023-07-29 03:01:58,334 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-29 03:01:58,345 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:01:58,345 DEBUG [RS:0;jenkins-hbase16:44149] zookeeper.ZKUtil(162): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,345 WARN [RS:0;jenkins-hbase16:44149] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-29 03:01:58,346 INFO [RS:0;jenkins-hbase16:44149] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 03:01:58,346 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,346 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,44149,1690599717958] 2023-07-29 03:01:58,350 DEBUG [RS:0;jenkins-hbase16:44149] zookeeper.ZKUtil(162): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,350 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-29 03:01:58,350 INFO [RS:0;jenkins-hbase16:44149] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-29 03:01:58,351 INFO [RS:0;jenkins-hbase16:44149] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-29 03:01:58,352 INFO [RS:0;jenkins-hbase16:44149] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-29 03:01:58,352 INFO [RS:0;jenkins-hbase16:44149] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,352 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-29 03:01:58,353 INFO [RS:0;jenkins-hbase16:44149] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,353 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 DEBUG [RS:0;jenkins-hbase16:44149] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-07-29 03:01:58,354 INFO [RS:0;jenkins-hbase16:44149] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,355 INFO [RS:0;jenkins-hbase16:44149] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,355 INFO [RS:0;jenkins-hbase16:44149] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,366 INFO [RS:0;jenkins-hbase16:44149] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-29 03:01:58,367 INFO [RS:0;jenkins-hbase16:44149] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44149,1690599717958-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,374 INFO [RS:0;jenkins-hbase16:44149] regionserver.Replication(203): jenkins-hbase16.apache.org,44149,1690599717958 started 2023-07-29 03:01:58,374 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1637): Serving as jenkins-hbase16.apache.org,44149,1690599717958, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:44149, sessionid=0x101af06b9f10001 2023-07-29 03:01:58,375 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-29 03:01:58,375 DEBUG [RS:0;jenkins-hbase16:44149] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,375 DEBUG [RS:0;jenkins-hbase16:44149] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,44149,1690599717958' 2023-07-29 03:01:58,375 DEBUG [RS:0;jenkins-hbase16:44149] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-29 03:01:58,375 DEBUG [RS:0;jenkins-hbase16:44149] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-29 03:01:58,376 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-29 03:01:58,376 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-29 03:01:58,376 DEBUG [RS:0;jenkins-hbase16:44149] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,376 DEBUG [RS:0;jenkins-hbase16:44149] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,44149,1690599717958' 2023-07-29 03:01:58,376 DEBUG [RS:0;jenkins-hbase16:44149] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-29 03:01:58,376 DEBUG [RS:0;jenkins-hbase16:44149] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-29 03:01:58,376 DEBUG [RS:0;jenkins-hbase16:44149] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-29 03:01:58,376 INFO [RS:0;jenkins-hbase16:44149] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-29 03:01:58,376 INFO [RS:0;jenkins-hbase16:44149] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-29 03:01:58,416 DEBUG [jenkins-hbase16:41697] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-29 03:01:58,417 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,44149,1690599717958, state=OPENING 2023-07-29 03:01:58,421 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-29 03:01:58,430 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:58,430 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,44149,1690599717958}] 2023-07-29 03:01:58,430 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 03:01:58,481 INFO [RS:0;jenkins-hbase16:44149] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C44149%2C1690599717958, suffix=, logDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/jenkins-hbase16.apache.org,44149,1690599717958, archiveDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/oldWALs, maxLogs=32 2023-07-29 03:01:58,498 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK] 2023-07-29 03:01:58,498 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK] 2023-07-29 03:01:58,500 INFO [RS:0;jenkins-hbase16:44149] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/jenkins-hbase16.apache.org,44149,1690599717958/jenkins-hbase16.apache.org%2C44149%2C1690599717958.1690599718482 2023-07-29 03:01:58,500 DEBUG [RS:0;jenkins-hbase16:44149] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK], DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK]] 2023-07-29 03:01:58,585 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,586 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-29 03:01:58,590 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:46744, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-29 03:01:58,597 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-29 03:01:58,597 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 03:01:58,599 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C44149%2C1690599717958.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/jenkins-hbase16.apache.org,44149,1690599717958, archiveDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/oldWALs, maxLogs=32 2023-07-29 03:01:58,615 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK] 2023-07-29 03:01:58,616 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK] 2023-07-29 03:01:58,618 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/jenkins-hbase16.apache.org,44149,1690599717958/jenkins-hbase16.apache.org%2C44149%2C1690599717958.meta.1690599718600.meta 2023-07-29 03:01:58,618 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK], DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK]] 2023-07-29 03:01:58,618 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-29 03:01:58,618 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-29 03:01:58,618 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-29 03:01:58,619 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-29 03:01:58,619 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-29 03:01:58,619 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:01:58,619 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-29 03:01:58,619 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-29 03:01:58,620 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-29 03:01:58,621 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/info 2023-07-29 03:01:58,621 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/info 2023-07-29 03:01:58,621 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-29 03:01:58,622 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:58,622 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-29 03:01:58,623 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/rep_barrier 2023-07-29 03:01:58,623 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/rep_barrier 2023-07-29 03:01:58,624 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-29 03:01:58,624 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:58,624 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-29 03:01:58,625 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/table 2023-07-29 03:01:58,625 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/table 2023-07-29 03:01:58,626 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-29 03:01:58,626 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:58,627 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740 2023-07-29 03:01:58,628 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740 2023-07-29 03:01:58,630 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-07-29 03:01:58,632 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-29 03:01:58,633 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=813304, jitterRate=0.03417019546031952}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-07-29 03:01:58,633 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-29 03:01:58,633 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690599718585 2023-07-29 03:01:58,638 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-29 03:01:58,638 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-29 03:01:58,639 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,44149,1690599717958, state=OPEN 2023-07-29 03:01:58,646 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-29 03:01:58,646 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-29 03:01:58,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-29 03:01:58,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,44149,1690599717958 in 216 msec 2023-07-29 03:01:58,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-29 03:01:58,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 385 msec 2023-07-29 03:01:58,650 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 430 msec 2023-07-29 03:01:58,650 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690599718650, completionTime=-1 2023-07-29 03:01:58,650 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-07-29 03:01:58,650 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-29 03:01:58,653 DEBUG [hconnection-0x5d5363e2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 03:01:58,654 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:46752, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 03:01:58,655 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-29 03:01:58,655 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690599778655 2023-07-29 03:01:58,655 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690599838655 2023-07-29 03:01:58,655 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-29 03:01:58,672 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41697,1690599717827-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,672 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41697,1690599717827-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,672 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41697,1690599717827-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,672 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:41697, period=300000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,672 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-29 03:01:58,672 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-29 03:01:58,672 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-29 03:01:58,673 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-29 03:01:58,675 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-29 03:01:58,675 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-29 03:01:58,676 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-29 03:01:58,679 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/.tmp/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:58,679 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/.tmp/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58 empty. 2023-07-29 03:01:58,680 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/.tmp/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:58,680 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-29 03:01:58,694 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-29 03:01:58,695 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => a289295cb93d0e4c4ab456f59f375d58, NAME => 'hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/.tmp 2023-07-29 03:01:58,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:01:58,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing a289295cb93d0e4c4ab456f59f375d58, disabling compactions & flushes 2023-07-29 03:01:58,705 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:58,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:58,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. after waiting 0 ms 2023-07-29 03:01:58,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:58,705 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:58,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for a289295cb93d0e4c4ab456f59f375d58: 2023-07-29 03:01:58,707 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-29 03:01:58,708 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599718708"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690599718708"}]},"ts":"1690599718708"} 2023-07-29 03:01:58,710 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-29 03:01:58,710 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-29 03:01:58,710 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599718710"}]},"ts":"1690599718710"} 2023-07-29 03:01:58,711 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-29 03:01:58,755 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a289295cb93d0e4c4ab456f59f375d58, ASSIGN}] 2023-07-29 03:01:58,757 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a289295cb93d0e4c4ab456f59f375d58, ASSIGN 2023-07-29 03:01:58,758 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=a289295cb93d0e4c4ab456f59f375d58, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,44149,1690599717958; forceNewPlan=false, retain=false 2023-07-29 03:01:58,910 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a289295cb93d0e4c4ab456f59f375d58, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:58,910 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599718910"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1690599718910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690599718910"}]},"ts":"1690599718910"} 2023-07-29 03:01:58,914 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure a289295cb93d0e4c4ab456f59f375d58, server=jenkins-hbase16.apache.org,44149,1690599717958}] 2023-07-29 03:01:59,074 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,074 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a289295cb93d0e4c4ab456f59f375d58, NAME => 'hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.', STARTKEY => '', ENDKEY => ''} 2023-07-29 03:01:59,074 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,074 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-29 03:01:59,075 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,075 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,077 INFO [StoreOpener-a289295cb93d0e4c4ab456f59f375d58-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,080 DEBUG [StoreOpener-a289295cb93d0e4c4ab456f59f375d58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/info 2023-07-29 03:01:59,080 DEBUG [StoreOpener-a289295cb93d0e4c4ab456f59f375d58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/info 2023-07-29 03:01:59,080 INFO [StoreOpener-a289295cb93d0e4c4ab456f59f375d58-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a289295cb93d0e4c4ab456f59f375d58 columnFamilyName info 2023-07-29 03:01:59,081 INFO [StoreOpener-a289295cb93d0e4c4ab456f59f375d58-1] regionserver.HStore(310): Store=a289295cb93d0e4c4ab456f59f375d58/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-29 03:01:59,083 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,083 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,086 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,088 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-29 03:01:59,089 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened a289295cb93d0e4c4ab456f59f375d58; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=775303, jitterRate=-0.014151692390441895}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-29 03:01:59,089 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for a289295cb93d0e4c4ab456f59f375d58: 2023-07-29 03:01:59,090 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58., pid=6, masterSystemTime=1690599719067 2023-07-29 03:01:59,091 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,091 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,091 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a289295cb93d0e4c4ab456f59f375d58, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:59,092 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690599719091"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1690599719091"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690599719091"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690599719091"}]},"ts":"1690599719091"} 2023-07-29 03:01:59,094 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-29 03:01:59,094 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure a289295cb93d0e4c4ab456f59f375d58, server=jenkins-hbase16.apache.org,44149,1690599717958 in 179 msec 2023-07-29 03:01:59,096 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-29 03:01:59,096 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=a289295cb93d0e4c4ab456f59f375d58, ASSIGN in 339 msec 2023-07-29 03:01:59,096 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-29 03:01:59,097 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690599719096"}]},"ts":"1690599719096"} 2023-07-29 03:01:59,098 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-29 03:01:59,130 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-29 03:01:59,132 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 458 msec 2023-07-29 03:01:59,175 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-29 03:01:59,180 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-29 03:01:59,180 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:59,185 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-29 03:01:59,204 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 03:01:59,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 30 msec 2023-07-29 03:01:59,217 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-29 03:01:59,229 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-29 03:01:59,241 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 22 msec 2023-07-29 03:01:59,263 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-29 03:01:59,279 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-29 03:01:59,280 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.274sec 2023-07-29 03:01:59,280 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-29 03:01:59,280 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-29 03:01:59,280 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-29 03:01:59,280 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41697,1690599717827-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-29 03:01:59,280 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41697,1690599717827-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-29 03:01:59,282 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-29 03:01:59,379 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ReadOnlyZKClient(139): Connect 0x4aec3aa4 to 127.0.0.1:64034 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-29 03:01:59,398 DEBUG [Listener at localhost.localdomain/46205] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68c048c9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-29 03:01:59,402 DEBUG [hconnection-0xcbc84c6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-29 03:01:59,405 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:46764, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-29 03:01:59,408 INFO [Listener at localhost.localdomain/46205] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:59,408 INFO [Listener at localhost.localdomain/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-29 03:01:59,430 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-29 03:01:59,430 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:59,432 INFO [Listener at localhost.localdomain/46205] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-07-29 03:01:59,432 INFO [Listener at localhost.localdomain/46205] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-29 03:01:59,436 INFO [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/test.com,8080,1, archiveDir=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/oldWALs, maxLogs=32 2023-07-29 03:01:59,455 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK] 2023-07-29 03:01:59,456 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK] 2023-07-29 03:01:59,458 INFO [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/test.com,8080,1/test.com%2C8080%2C1.1690599719436 2023-07-29 03:01:59,458 DEBUG [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK], DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK]] 2023-07-29 03:01:59,470 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK] 2023-07-29 03:01:59,470 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK] 2023-07-29 03:01:59,472 INFO [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/test.com,8080,1/test.com%2C8080%2C1.1690599719436 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/test.com,8080,1/test.com%2C8080%2C1.1690599719458 2023-07-29 03:01:59,472 DEBUG [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32795,DS-01c0b669-5793-408a-a5b4-1a2ccea88731,DISK], DatanodeInfoWithStorage[127.0.0.1:45725,DS-9ce1620b-7955-4a95-8632-73c3d2eddf01,DISK]] 2023-07-29 03:01:59,472 DEBUG [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(716): hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/test.com,8080,1/test.com%2C8080%2C1.1690599719436 is not closed yet, will try archiving it next time 2023-07-29 03:01:59,478 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/WALs/test.com,8080,1/test.com%2C8080%2C1.1690599719436 to hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/oldWALs/test.com%2C8080%2C1.1690599719436 2023-07-29 03:01:59,480 DEBUG [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/oldWALs 2023-07-29 03:01:59,481 INFO [Listener at localhost.localdomain/46205] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1690599719458) 2023-07-29 03:01:59,481 INFO [Listener at localhost.localdomain/46205] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-29 03:01:59,481 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4aec3aa4 to 127.0.0.1:64034 2023-07-29 03:01:59,481 DEBUG [Listener at localhost.localdomain/46205] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:59,482 DEBUG [Listener at localhost.localdomain/46205] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-29 03:01:59,482 DEBUG [Listener at localhost.localdomain/46205] util.JVMClusterUtil(257): Found active master hash=1474424712, stopped=false 2023-07-29 03:01:59,482 INFO [Listener at localhost.localdomain/46205] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:59,496 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 03:01:59,496 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-29 03:01:59,496 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:59,496 INFO [Listener at localhost.localdomain/46205] procedure2.ProcedureExecutor(629): Stopping 2023-07-29 03:01:59,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:01:59,498 DEBUG [Listener at localhost.localdomain/46205] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7590d15f to 127.0.0.1:64034 2023-07-29 03:01:59,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-29 03:01:59,498 DEBUG [Listener at localhost.localdomain/46205] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:59,499 INFO [Listener at localhost.localdomain/46205] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,44149,1690599717958' ***** 2023-07-29 03:01:59,499 INFO [Listener at localhost.localdomain/46205] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-29 03:01:59,499 INFO [RS:0;jenkins-hbase16:44149] regionserver.HeapMemoryManager(220): Stopping 2023-07-29 03:01:59,499 INFO [RS:0;jenkins-hbase16:44149] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-29 03:01:59,499 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-29 03:01:59,499 INFO [RS:0;jenkins-hbase16:44149] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-29 03:01:59,500 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(3305): Received CLOSE for a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,500 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:59,500 DEBUG [RS:0;jenkins-hbase16:44149] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a08acb4 to 127.0.0.1:64034 2023-07-29 03:01:59,500 DEBUG [RS:0;jenkins-hbase16:44149] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:59,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing a289295cb93d0e4c4ab456f59f375d58, disabling compactions & flushes 2023-07-29 03:01:59,501 INFO [RS:0;jenkins-hbase16:44149] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-29 03:01:59,501 INFO [RS:0;jenkins-hbase16:44149] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-29 03:01:59,501 INFO [RS:0;jenkins-hbase16:44149] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-29 03:01:59,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. after waiting 0 ms 2023-07-29 03:01:59,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,501 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-29 03:01:59,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing a289295cb93d0e4c4ab456f59f375d58 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-29 03:01:59,501 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-29 03:01:59,502 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, a289295cb93d0e4c4ab456f59f375d58=hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58.} 2023-07-29 03:01:59,502 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-29 03:01:59,502 DEBUG [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1504): Waiting on 1588230740, a289295cb93d0e4c4ab456f59f375d58 2023-07-29 03:01:59,502 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-29 03:01:59,502 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-29 03:01:59,502 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-29 03:01:59,502 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-29 03:01:59,503 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-07-29 03:01:59,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/.tmp/info/e5957a7f54e7413d8361236a44f2c4fa 2023-07-29 03:01:59,516 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/.tmp/info/252293886a754eb5a7fb4a36a0299979 2023-07-29 03:01:59,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/.tmp/info/e5957a7f54e7413d8361236a44f2c4fa as hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/info/e5957a7f54e7413d8361236a44f2c4fa 2023-07-29 03:01:59,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/info/e5957a7f54e7413d8361236a44f2c4fa, entries=2, sequenceid=6, filesize=4.8 K 2023-07-29 03:01:59,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a289295cb93d0e4c4ab456f59f375d58 in 28ms, sequenceid=6, compaction requested=false 2023-07-29 03:01:59,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-29 03:01:59,532 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/.tmp/table/5fe98dcd8ff04eefb8eb7c2bd033bcfa 2023-07-29 03:01:59,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/namespace/a289295cb93d0e4c4ab456f59f375d58/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-29 03:01:59,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for a289295cb93d0e4c4ab456f59f375d58: 2023-07-29 03:01:59,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690599718672.a289295cb93d0e4c4ab456f59f375d58. 2023-07-29 03:01:59,537 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/.tmp/info/252293886a754eb5a7fb4a36a0299979 as hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/info/252293886a754eb5a7fb4a36a0299979 2023-07-29 03:01:59,543 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/info/252293886a754eb5a7fb4a36a0299979, entries=10, sequenceid=9, filesize=5.9 K 2023-07-29 03:01:59,544 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/.tmp/table/5fe98dcd8ff04eefb8eb7c2bd033bcfa as hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/table/5fe98dcd8ff04eefb8eb7c2bd033bcfa 2023-07-29 03:01:59,549 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/table/5fe98dcd8ff04eefb8eb7c2bd033bcfa, entries=2, sequenceid=9, filesize=4.7 K 2023-07-29 03:01:59,549 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1292, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 46ms, sequenceid=9, compaction requested=false 2023-07-29 03:01:59,549 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-29 03:01:59,560 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-07-29 03:01:59,561 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-29 03:01:59,561 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-29 03:01:59,561 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-29 03:01:59,561 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-29 03:01:59,702 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,44149,1690599717958; all regions closed. 2023-07-29 03:01:59,711 DEBUG [RS:0;jenkins-hbase16:44149] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/oldWALs 2023-07-29 03:01:59,711 INFO [RS:0;jenkins-hbase16:44149] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C44149%2C1690599717958.meta:.meta(num 1690599718600) 2023-07-29 03:01:59,716 DEBUG [RS:0;jenkins-hbase16:44149] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/oldWALs 2023-07-29 03:01:59,717 INFO [RS:0;jenkins-hbase16:44149] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C44149%2C1690599717958:(num 1690599718482) 2023-07-29 03:01:59,717 DEBUG [RS:0;jenkins-hbase16:44149] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:59,717 INFO [RS:0;jenkins-hbase16:44149] regionserver.LeaseManager(133): Closed leases 2023-07-29 03:01:59,717 INFO [RS:0;jenkins-hbase16:44149] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-29 03:01:59,717 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 03:01:59,718 INFO [RS:0;jenkins-hbase16:44149] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:44149 2023-07-29 03:01:59,729 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:01:59,729 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,44149,1690599717958 2023-07-29 03:01:59,730 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-29 03:01:59,730 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,44149,1690599717958] 2023-07-29 03:01:59,730 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,44149,1690599717958; numProcessing=1 2023-07-29 03:01:59,746 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,44149,1690599717958 already deleted, retry=false 2023-07-29 03:01:59,746 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,44149,1690599717958 expired; onlineServers=0 2023-07-29 03:01:59,746 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,41697,1690599717827' ***** 2023-07-29 03:01:59,746 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-29 03:01:59,747 DEBUG [M:0;jenkins-hbase16:41697] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53de71ac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-07-29 03:01:59,747 INFO [M:0;jenkins-hbase16:41697] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,41697,1690599717827 2023-07-29 03:01:59,748 INFO [M:0;jenkins-hbase16:41697] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,41697,1690599717827; all regions closed. 2023-07-29 03:01:59,748 DEBUG [M:0;jenkins-hbase16:41697] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-29 03:01:59,748 DEBUG [M:0;jenkins-hbase16:41697] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-29 03:01:59,748 DEBUG [M:0;jenkins-hbase16:41697] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-29 03:01:59,748 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-29 03:01:59,750 INFO [M:0;jenkins-hbase16:41697] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-29 03:01:59,750 INFO [M:0;jenkins-hbase16:41697] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-29 03:01:59,750 INFO [M:0;jenkins-hbase16:41697] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-07-29 03:01:59,750 DEBUG [M:0;jenkins-hbase16:41697] master.HMaster(1512): Stopping service threads 2023-07-29 03:01:59,750 INFO [M:0;jenkins-hbase16:41697] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-29 03:01:59,748 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599718227] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1690599718227,5,FailOnTimeoutGroup] 2023-07-29 03:01:59,748 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599718227] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1690599718227,5,FailOnTimeoutGroup] 2023-07-29 03:01:59,751 ERROR [M:0;jenkins-hbase16:41697] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-07-29 03:01:59,753 INFO [M:0;jenkins-hbase16:41697] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-29 03:01:59,753 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-29 03:01:59,754 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-29 03:01:59,755 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-29 03:01:59,755 DEBUG [M:0;jenkins-hbase16:41697] zookeeper.ZKUtil(398): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-29 03:01:59,755 WARN [M:0;jenkins-hbase16:41697] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-29 03:01:59,755 INFO [M:0;jenkins-hbase16:41697] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-29 03:01:59,756 INFO [M:0;jenkins-hbase16:41697] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-29 03:01:59,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-29 03:01:59,757 DEBUG [M:0;jenkins-hbase16:41697] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-29 03:01:59,757 INFO [M:0;jenkins-hbase16:41697] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:59,757 DEBUG [M:0;jenkins-hbase16:41697] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:59,757 DEBUG [M:0;jenkins-hbase16:41697] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-29 03:01:59,757 DEBUG [M:0;jenkins-hbase16:41697] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:59,757 INFO [M:0;jenkins-hbase16:41697] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.09 KB heapSize=29.59 KB 2023-07-29 03:01:59,770 INFO [M:0;jenkins-hbase16:41697] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.09 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/98126b6b5b38460498701a10c16b0f81 2023-07-29 03:01:59,774 DEBUG [M:0;jenkins-hbase16:41697] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/98126b6b5b38460498701a10c16b0f81 as hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/98126b6b5b38460498701a10c16b0f81 2023-07-29 03:01:59,778 INFO [M:0;jenkins-hbase16:41697] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44797/user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/98126b6b5b38460498701a10c16b0f81, entries=8, sequenceid=66, filesize=6.3 K 2023-07-29 03:01:59,779 INFO [M:0;jenkins-hbase16:41697] regionserver.HRegion(2948): Finished flush of dataSize ~24.09 KB/24669, heapSize ~29.57 KB/30280, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=66, compaction requested=false 2023-07-29 03:01:59,780 INFO [M:0;jenkins-hbase16:41697] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-29 03:01:59,780 DEBUG [M:0;jenkins-hbase16:41697] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-29 03:01:59,782 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/8ee1941e-e6af-b221-53ca-85ad0a14a092/MasterData/WALs/jenkins-hbase16.apache.org,41697,1690599717827/jenkins-hbase16.apache.org%2C41697%2C1690599717827.1690599718075 not finished, retry = 0 2023-07-29 03:01:59,838 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:01:59,838 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:44149-0x101af06b9f10001, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:01:59,838 INFO [RS:0;jenkins-hbase16:44149] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,44149,1690599717958; zookeeper connection closed. 2023-07-29 03:01:59,839 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4b8b5eff] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4b8b5eff 2023-07-29 03:01:59,839 INFO [Listener at localhost.localdomain/46205] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-29 03:01:59,884 INFO [M:0;jenkins-hbase16:41697] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-29 03:01:59,884 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-29 03:01:59,885 INFO [M:0;jenkins-hbase16:41697] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:41697 2023-07-29 03:01:59,896 DEBUG [M:0;jenkins-hbase16:41697] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,41697,1690599717827 already deleted, retry=false 2023-07-29 03:02:00,005 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:02:00,005 DEBUG [Listener at localhost.localdomain/46205-EventThread] zookeeper.ZKWatcher(600): master:41697-0x101af06b9f10000, quorum=127.0.0.1:64034, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-29 03:02:00,005 INFO [M:0;jenkins-hbase16:41697] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,41697,1690599717827; zookeeper connection closed. 2023-07-29 03:02:00,010 WARN [Listener at localhost.localdomain/46205] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 03:02:00,045 INFO [Listener at localhost.localdomain/46205] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 03:02:00,152 WARN [BP-1684749260-188.40.62.62-1690599716455 heartbeating to localhost.localdomain/127.0.0.1:44797] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 03:02:00,153 WARN [BP-1684749260-188.40.62.62-1690599716455 heartbeating to localhost.localdomain/127.0.0.1:44797] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1684749260-188.40.62.62-1690599716455 (Datanode Uuid 376964fd-866a-4a1f-8b7e-da1ab06ea1d1) service to localhost.localdomain/127.0.0.1:44797 2023-07-29 03:02:00,154 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1/dfs/data/data3/current/BP-1684749260-188.40.62.62-1690599716455] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:02:00,155 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1/dfs/data/data4/current/BP-1684749260-188.40.62.62-1690599716455] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:02:00,157 WARN [Listener at localhost.localdomain/46205] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-29 03:02:00,161 INFO [Listener at localhost.localdomain/46205] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-29 03:02:00,266 WARN [BP-1684749260-188.40.62.62-1690599716455 heartbeating to localhost.localdomain/127.0.0.1:44797] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-29 03:02:00,266 WARN [BP-1684749260-188.40.62.62-1690599716455 heartbeating to localhost.localdomain/127.0.0.1:44797] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1684749260-188.40.62.62-1690599716455 (Datanode Uuid 04e4a617-7874-427d-a4dc-3f6dac385920) service to localhost.localdomain/127.0.0.1:44797 2023-07-29 03:02:00,268 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1/dfs/data/data1/current/BP-1684749260-188.40.62.62-1690599716455] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:02:00,269 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4fe436b6-1e64-2ccf-bbc1-f9d9a1a801e1/cluster_115f7121-50b5-8f7a-90f7-cbf0d3b340c1/dfs/data/data2/current/BP-1684749260-188.40.62.62-1690599716455] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-29 03:02:00,281 INFO [Listener at localhost.localdomain/46205] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-07-29 03:02:00,355 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-29 03:02:00,398 INFO [Listener at localhost.localdomain/46205] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-29 03:02:00,408 INFO [Listener at localhost.localdomain/46205] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-29 03:02:00,416 INFO [Listener at localhost.localdomain/46205] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=100 (was 77) - Thread LEAK? -, OpenFileDescriptor=482 (was 467) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=49 (was 36) - SystemLoadAverage LEAK? -, ProcessCount=165 (was 165), AvailableMemoryMB=5605 (was 5612)