2023-08-07 14:56:54,078 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac 2023-08-07 14:56:54,092 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-08-07 14:56:54,126 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=450, ProcessCount=171, AvailableMemoryMB=1942 2023-08-07 14:56:54,132 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-07 14:56:54,133 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d, deleteOnExit=true 2023-08-07 14:56:54,133 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-07 14:56:54,134 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/test.cache.data in system properties and HBase conf 2023-08-07 14:56:54,134 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/hadoop.tmp.dir in system properties and HBase conf 2023-08-07 14:56:54,135 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/hadoop.log.dir in system properties and HBase conf 2023-08-07 14:56:54,135 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-07 14:56:54,135 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-07 14:56:54,135 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-07 14:56:54,255 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-07 14:56:54,647 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-07 14:56:54,651 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-07 14:56:54,651 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-07 14:56:54,651 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-07 14:56:54,652 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 14:56:54,652 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-07 14:56:54,652 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-07 14:56:54,653 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 14:56:54,653 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 14:56:54,653 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-07 14:56:54,653 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/nfs.dump.dir in system properties and HBase conf 2023-08-07 14:56:54,654 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir in system properties and HBase conf 2023-08-07 14:56:54,654 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 14:56:54,654 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-07 14:56:54,654 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-07 14:56:55,185 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 14:56:55,189 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 14:56:55,480 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-07 14:56:55,647 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-07 14:56:55,667 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:56:55,710 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:56:55,745 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir/Jetty_localhost_41681_hdfs____pkw4ak/webapp 2023-08-07 14:56:55,900 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41681 2023-08-07 14:56:55,919 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 14:56:55,919 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 14:56:56,466 WARN [Listener at localhost/44501] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:56:56,529 WARN [Listener at localhost/44501] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:56:56,546 WARN [Listener at localhost/44501] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:56:56,552 INFO [Listener at localhost/44501] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:56:56,556 INFO [Listener at localhost/44501] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir/Jetty_localhost_41361_datanode____oh2qk8/webapp 2023-08-07 14:56:56,655 INFO [Listener at localhost/44501] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41361 2023-08-07 14:56:56,950 WARN [Listener at localhost/45569] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:56:56,960 WARN [Listener at localhost/45569] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:56:56,964 WARN [Listener at localhost/45569] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:56:56,965 INFO [Listener at localhost/45569] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:56:56,970 INFO [Listener at localhost/45569] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir/Jetty_localhost_42319_datanode____j0mfus/webapp 2023-08-07 14:56:57,076 INFO [Listener at localhost/45569] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42319 2023-08-07 14:56:57,089 WARN [Listener at localhost/42441] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:56:57,385 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc519eacfba11c6d6: Processing first storage report for DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5 from datanode 63b7ec26-c108-4c34-b46b-bdea9dc84261 2023-08-07 14:56:57,386 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc519eacfba11c6d6: from storage DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5 node DatanodeRegistration(127.0.0.1:46283, datanodeUuid=63b7ec26-c108-4c34-b46b-bdea9dc84261, infoPort=43303, infoSecurePort=0, ipcPort=42441, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-07 14:56:57,387 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc05e8a00f5d5021e: Processing first storage report for DS-a415f36a-df11-4568-911a-a2c1fc57e0b0 from datanode 8beb4d85-3072-419e-b9cc-cf1c03909e14 2023-08-07 14:56:57,387 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc05e8a00f5d5021e: from storage DS-a415f36a-df11-4568-911a-a2c1fc57e0b0 node DatanodeRegistration(127.0.0.1:39221, datanodeUuid=8beb4d85-3072-419e-b9cc-cf1c03909e14, infoPort=41241, infoSecurePort=0, ipcPort=45569, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:56:57,387 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc519eacfba11c6d6: Processing first storage report for DS-6b6c1f86-3644-45fc-bd90-ac36bbd99007 from datanode 63b7ec26-c108-4c34-b46b-bdea9dc84261 2023-08-07 14:56:57,387 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc519eacfba11c6d6: from storage DS-6b6c1f86-3644-45fc-bd90-ac36bbd99007 node DatanodeRegistration(127.0.0.1:46283, datanodeUuid=63b7ec26-c108-4c34-b46b-bdea9dc84261, infoPort=43303, infoSecurePort=0, ipcPort=42441, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:56:57,387 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc05e8a00f5d5021e: Processing first storage report for DS-9ed59e53-e4f4-482c-a31c-377ebae8e89e from datanode 8beb4d85-3072-419e-b9cc-cf1c03909e14 2023-08-07 14:56:57,387 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc05e8a00f5d5021e: from storage DS-9ed59e53-e4f4-482c-a31c-377ebae8e89e node DatanodeRegistration(127.0.0.1:39221, datanodeUuid=8beb4d85-3072-419e-b9cc-cf1c03909e14, infoPort=41241, infoSecurePort=0, ipcPort=45569, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:56:57,498 DEBUG [Listener at localhost/42441] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac 2023-08-07 14:56:57,562 INFO [Listener at localhost/42441] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/zookeeper_0, clientPort=57140, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-07 14:56:57,577 INFO [Listener at localhost/42441] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57140 2023-08-07 14:56:57,587 INFO [Listener at localhost/42441] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:56:57,589 INFO [Listener at localhost/42441] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:56:58,300 INFO [Listener at localhost/42441] util.FSUtils(471): Created version file at hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4 with version=8 2023-08-07 14:56:58,300 INFO [Listener at localhost/42441] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/hbase-staging 2023-08-07 14:56:58,672 INFO [Listener at localhost/42441] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-07 14:56:59,336 INFO [Listener at localhost/42441] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 14:56:59,373 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:56:59,373 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 14:56:59,374 INFO [Listener at localhost/42441] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 14:56:59,374 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:56:59,374 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 14:56:59,545 INFO [Listener at localhost/42441] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 14:56:59,634 DEBUG [Listener at localhost/42441] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-07 14:56:59,743 INFO [Listener at localhost/42441] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41067 2023-08-07 14:56:59,754 INFO [Listener at localhost/42441] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:56:59,757 INFO [Listener at localhost/42441] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:56:59,780 INFO [Listener at localhost/42441] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41067 connecting to ZooKeeper ensemble=127.0.0.1:57140 2023-08-07 14:56:59,843 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:410670x0, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 14:56:59,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41067-0x101dfeed3ce0000 connected 2023-08-07 14:56:59,876 DEBUG [Listener at localhost/42441] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:56:59,877 DEBUG [Listener at localhost/42441] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:56:59,881 DEBUG [Listener at localhost/42441] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 14:56:59,890 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41067 2023-08-07 14:56:59,890 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41067 2023-08-07 14:56:59,891 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41067 2023-08-07 14:56:59,891 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41067 2023-08-07 14:56:59,892 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41067 2023-08-07 14:56:59,899 INFO [Listener at localhost/42441] master.HMaster(444): hbase.rootdir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4, hbase.cluster.distributed=false 2023-08-07 14:56:59,976 INFO [Listener at localhost/42441] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 14:56:59,977 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:56:59,977 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 14:56:59,977 INFO [Listener at localhost/42441] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 14:56:59,977 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:56:59,977 INFO [Listener at localhost/42441] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 14:56:59,983 INFO [Listener at localhost/42441] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 14:56:59,986 INFO [Listener at localhost/42441] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41313 2023-08-07 14:56:59,988 INFO [Listener at localhost/42441] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 14:56:59,995 DEBUG [Listener at localhost/42441] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 14:56:59,997 INFO [Listener at localhost/42441] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:56:59,999 INFO [Listener at localhost/42441] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:00,001 INFO [Listener at localhost/42441] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41313 connecting to ZooKeeper ensemble=127.0.0.1:57140 2023-08-07 14:57:00,005 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:413130x0, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 14:57:00,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41313-0x101dfeed3ce0001 connected 2023-08-07 14:57:00,006 DEBUG [Listener at localhost/42441] zookeeper.ZKUtil(164): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:57:00,008 DEBUG [Listener at localhost/42441] zookeeper.ZKUtil(164): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:57:00,009 DEBUG [Listener at localhost/42441] zookeeper.ZKUtil(164): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 14:57:00,009 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41313 2023-08-07 14:57:00,010 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41313 2023-08-07 14:57:00,011 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41313 2023-08-07 14:57:00,011 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41313 2023-08-07 14:57:00,012 DEBUG [Listener at localhost/42441] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41313 2023-08-07 14:57:00,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:00,024 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 14:57:00,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:00,049 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 14:57:00,049 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 14:57:00,049 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:00,050 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 14:57:00,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 14:57:00,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,41067,1691420218479 from backup master directory 2023-08-07 14:57:00,057 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:00,057 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 14:57:00,058 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 14:57:00,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:00,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-07 14:57:00,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-07 14:57:00,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/hbase.id with ID: 6783188e-4625-4b2a-8bbc-8630bfcd13cc 2023-08-07 14:57:00,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:00,246 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:00,314 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x0b154b7e to 127.0.0.1:57140 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:57:00,352 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a9e2f22, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:57:00,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:00,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 14:57:00,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:57:00,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store-tmp 2023-08-07 14:57:00,470 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:00,470 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 14:57:00,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:00,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:00,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 14:57:00,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:00,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:00,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:57:00,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:00,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41067%2C1691420218479, suffix=, logDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479, archiveDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/oldWALs, maxLogs=10 2023-08-07 14:57:00,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-07 14:57:00,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-07 14:57:00,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:00,582 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:00,584 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46283,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK] 2023-08-07 14:57:00,584 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39221,DS-a415f36a-df11-4568-911a-a2c1fc57e0b0,DISK] 2023-08-07 14:57:00,595 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:00,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 2023-08-07 14:57:00,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46283,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK], DatanodeInfoWithStorage[127.0.0.1:39221,DS-a415f36a-df11-4568-911a-a2c1fc57e0b0,DISK]] 2023-08-07 14:57:00,670 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:00,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:00,675 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:00,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:00,736 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:00,744 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 14:57:00,777 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 14:57:00,791 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:00,799 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:00,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:00,819 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:00,824 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:00,825 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=710445, jitterRate=-0.09662342071533203}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:57:00,825 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:57:00,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 14:57:00,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 14:57:00,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 14:57:00,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 14:57:00,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-07 14:57:00,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 38 msec 2023-08-07 14:57:00,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 14:57:00,918 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-07 14:57:00,923 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-07 14:57:00,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 14:57:00,960 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 14:57:00,962 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-07 14:57:00,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 14:57:00,975 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-07 14:57:00,977 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:00,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-07 14:57:00,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-07 14:57:00,991 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-07 14:57:00,995 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 14:57:00,995 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 14:57:00,995 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:00,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41067,1691420218479, sessionid=0x101dfeed3ce0000, setting cluster-up flag (Was=false) 2023-08-07 14:57:01,011 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:01,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-07 14:57:01,018 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:01,023 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:01,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-07 14:57:01,031 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:01,033 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.hbase-snapshot/.tmp 2023-08-07 14:57:01,116 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(951): ClusterId : 6783188e-4625-4b2a-8bbc-8630bfcd13cc 2023-08-07 14:57:01,121 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 14:57:01,126 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 14:57:01,127 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 14:57:01,131 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 14:57:01,132 DEBUG [RS:0;jenkins-hbase4:41313] zookeeper.ReadOnlyZKClient(139): Connect 0x20097838 to 127.0.0.1:57140 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:57:01,137 DEBUG [RS:0;jenkins-hbase4:41313] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ece9249, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:57:01,137 DEBUG [RS:0;jenkins-hbase4:41313] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2af11348, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 14:57:01,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-07 14:57:01,160 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41313 2023-08-07 14:57:01,164 INFO [RS:0;jenkins-hbase4:41313] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 14:57:01,164 INFO [RS:0;jenkins-hbase4:41313] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 14:57:01,164 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 14:57:01,166 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41067,1691420218479 with isa=jenkins-hbase4.apache.org/172.31.14.131:41313, startcode=1691420219975 2023-08-07 14:57:01,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:01,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:01,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:01,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:01,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 14:57:01,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 14:57:01,170 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691420251171 2023-08-07 14:57:01,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 14:57:01,179 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 14:57:01,179 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-07 14:57:01,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 14:57:01,184 DEBUG [RS:0;jenkins-hbase4:41313] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 14:57:01,184 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:01,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 14:57:01,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 14:57:01,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 14:57:01,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 14:57:01,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 14:57:01,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 14:57:01,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 14:57:01,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 14:57:01,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 14:57:01,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420221199,5,FailOnTimeoutGroup] 2023-08-07 14:57:01,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420221200,5,FailOnTimeoutGroup] 2023-08-07 14:57:01,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 14:57:01,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,232 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:01,233 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:01,234 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4 2023-08-07 14:57:01,251 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40365, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 14:57:01,259 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:01,262 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 14:57:01,265 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41067] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:01,265 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/info 2023-08-07 14:57:01,266 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 14:57:01,268 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:01,268 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 14:57:01,270 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:57:01,271 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 14:57:01,272 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:01,272 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 14:57:01,274 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/table 2023-08-07 14:57:01,275 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 14:57:01,275 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:01,277 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740 2023-08-07 14:57:01,278 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740 2023-08-07 14:57:01,282 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 14:57:01,284 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4 2023-08-07 14:57:01,284 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:44501 2023-08-07 14:57:01,284 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 14:57:01,284 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 14:57:01,288 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:01,290 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=866428, jitterRate=0.10172136127948761}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 14:57:01,290 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 14:57:01,290 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 14:57:01,290 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 14:57:01,290 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 14:57:01,290 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 14:57:01,290 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 14:57:01,291 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 14:57:01,291 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 14:57:01,291 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 14:57:01,292 DEBUG [RS:0;jenkins-hbase4:41313] zookeeper.ZKUtil(162): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:01,292 WARN [RS:0;jenkins-hbase4:41313] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 14:57:01,293 INFO [RS:0;jenkins-hbase4:41313] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:57:01,293 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1948): logDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:01,294 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41313,1691420219975] 2023-08-07 14:57:01,298 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 14:57:01,298 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-07 14:57:01,304 DEBUG [RS:0;jenkins-hbase4:41313] zookeeper.ZKUtil(162): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:01,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 14:57:01,315 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-07 14:57:01,318 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 14:57:01,321 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 14:57:01,324 INFO [RS:0;jenkins-hbase4:41313] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 14:57:01,342 INFO [RS:0;jenkins-hbase4:41313] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 14:57:01,345 INFO [RS:0;jenkins-hbase4:41313] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 14:57:01,346 INFO [RS:0;jenkins-hbase4:41313] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,346 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 14:57:01,353 INFO [RS:0;jenkins-hbase4:41313] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,353 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,354 DEBUG [RS:0;jenkins-hbase4:41313] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:01,355 INFO [RS:0;jenkins-hbase4:41313] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,355 INFO [RS:0;jenkins-hbase4:41313] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,355 INFO [RS:0;jenkins-hbase4:41313] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,373 INFO [RS:0;jenkins-hbase4:41313] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 14:57:01,375 INFO [RS:0;jenkins-hbase4:41313] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41313,1691420219975-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,390 INFO [RS:0;jenkins-hbase4:41313] regionserver.Replication(203): jenkins-hbase4.apache.org,41313,1691420219975 started 2023-08-07 14:57:01,390 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41313,1691420219975, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41313, sessionid=0x101dfeed3ce0001 2023-08-07 14:57:01,390 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 14:57:01,390 DEBUG [RS:0;jenkins-hbase4:41313] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:01,390 DEBUG [RS:0;jenkins-hbase4:41313] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41313,1691420219975' 2023-08-07 14:57:01,390 DEBUG [RS:0;jenkins-hbase4:41313] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:57:01,391 DEBUG [RS:0;jenkins-hbase4:41313] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:01,391 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 14:57:01,391 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 14:57:01,391 DEBUG [RS:0;jenkins-hbase4:41313] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:01,391 DEBUG [RS:0;jenkins-hbase4:41313] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41313,1691420219975' 2023-08-07 14:57:01,391 DEBUG [RS:0;jenkins-hbase4:41313] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-07 14:57:01,392 DEBUG [RS:0;jenkins-hbase4:41313] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-07 14:57:01,392 DEBUG [RS:0;jenkins-hbase4:41313] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 14:57:01,392 INFO [RS:0;jenkins-hbase4:41313] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 14:57:01,392 INFO [RS:0;jenkins-hbase4:41313] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 14:57:01,473 DEBUG [jenkins-hbase4:41067] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 14:57:01,476 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41313,1691420219975, state=OPENING 2023-08-07 14:57:01,484 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-07 14:57:01,485 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:01,485 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 14:57:01,489 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41313,1691420219975}] 2023-08-07 14:57:01,504 INFO [RS:0;jenkins-hbase4:41313] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41313%2C1691420219975, suffix=, logDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975, archiveDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/oldWALs, maxLogs=32 2023-08-07 14:57:01,524 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46283,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK] 2023-08-07 14:57:01,524 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39221,DS-a415f36a-df11-4568-911a-a2c1fc57e0b0,DISK] 2023-08-07 14:57:01,528 INFO [RS:0;jenkins-hbase4:41313] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 2023-08-07 14:57:01,528 DEBUG [RS:0;jenkins-hbase4:41313] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46283,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK], DatanodeInfoWithStorage[127.0.0.1:39221,DS-a415f36a-df11-4568-911a-a2c1fc57e0b0,DISK]] 2023-08-07 14:57:01,674 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:01,677 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 14:57:01,680 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41994, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 14:57:01,692 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 14:57:01,693 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:57:01,696 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta, suffix=.meta, logDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975, archiveDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/oldWALs, maxLogs=32 2023-08-07 14:57:01,713 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46283,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK] 2023-08-07 14:57:01,713 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39221,DS-a415f36a-df11-4568-911a-a2c1fc57e0b0,DISK] 2023-08-07 14:57:01,717 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta 2023-08-07 14:57:01,717 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46283,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK], DatanodeInfoWithStorage[127.0.0.1:39221,DS-a415f36a-df11-4568-911a-a2c1fc57e0b0,DISK]] 2023-08-07 14:57:01,718 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:01,720 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 14:57:01,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 14:57:01,745 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 14:57:01,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 14:57:01,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:01,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 14:57:01,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 14:57:01,755 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 14:57:01,757 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/info 2023-08-07 14:57:01,757 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/info 2023-08-07 14:57:01,758 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 14:57:01,759 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:01,759 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 14:57:01,760 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:57:01,760 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:57:01,761 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 14:57:01,762 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:01,762 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 14:57:01,763 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/table 2023-08-07 14:57:01,763 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/table 2023-08-07 14:57:01,764 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 14:57:01,764 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:01,766 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740 2023-08-07 14:57:01,769 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740 2023-08-07 14:57:01,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 14:57:01,776 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 14:57:01,778 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=711756, jitterRate=-0.09495548903942108}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 14:57:01,778 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 14:57:01,794 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691420221667 2023-08-07 14:57:01,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 14:57:01,812 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 14:57:01,813 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41313,1691420219975, state=OPEN 2023-08-07 14:57:01,816 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-07 14:57:01,816 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 14:57:01,820 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-07 14:57:01,821 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41313,1691420219975 in 327 msec 2023-08-07 14:57:01,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-07 14:57:01,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 514 msec 2023-08-07 14:57:01,831 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 749 msec 2023-08-07 14:57:01,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691420221831, completionTime=-1 2023-08-07 14:57:01,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-07 14:57:01,832 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 14:57:01,907 DEBUG [hconnection-0x5b13fea1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 14:57:01,911 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42000, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 14:57:01,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-07 14:57:01,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691420281932 2023-08-07 14:57:01,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691420341932 2023-08-07 14:57:01,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 100 msec 2023-08-07 14:57:01,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41067,1691420218479-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41067,1691420218479-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41067,1691420218479-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41067, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:01,963 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-07 14:57:01,971 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-07 14:57:01,972 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:01,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-07 14:57:01,985 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 14:57:01,988 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 14:57:02,013 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,015 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea empty. 2023-08-07 14:57:02,016 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,016 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-07 14:57:02,043 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:02,045 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => e4c09ac0e270308abcead3a999ce0bea, NAME => 'hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp 2023-08-07 14:57:02,061 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:02,061 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing e4c09ac0e270308abcead3a999ce0bea, disabling compactions & flushes 2023-08-07 14:57:02,061 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:02,061 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:02,061 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. after waiting 0 ms 2023-08-07 14:57:02,061 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:02,061 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:02,061 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for e4c09ac0e270308abcead3a999ce0bea: 2023-08-07 14:57:02,065 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 14:57:02,081 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420222068"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420222068"}]},"ts":"1691420222068"} 2023-08-07 14:57:02,106 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 14:57:02,108 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 14:57:02,112 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420222108"}]},"ts":"1691420222108"} 2023-08-07 14:57:02,115 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-07 14:57:02,125 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e4c09ac0e270308abcead3a999ce0bea, ASSIGN}] 2023-08-07 14:57:02,128 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e4c09ac0e270308abcead3a999ce0bea, ASSIGN 2023-08-07 14:57:02,129 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e4c09ac0e270308abcead3a999ce0bea, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41313,1691420219975; forceNewPlan=false, retain=false 2023-08-07 14:57:02,280 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e4c09ac0e270308abcead3a999ce0bea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:02,281 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420222280"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420222280"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420222280"}]},"ts":"1691420222280"} 2023-08-07 14:57:02,286 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure e4c09ac0e270308abcead3a999ce0bea, server=jenkins-hbase4.apache.org,41313,1691420219975}] 2023-08-07 14:57:02,447 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:02,448 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e4c09ac0e270308abcead3a999ce0bea, NAME => 'hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:02,450 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,450 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:02,451 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,451 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,453 INFO [StoreOpener-e4c09ac0e270308abcead3a999ce0bea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,455 DEBUG [StoreOpener-e4c09ac0e270308abcead3a999ce0bea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/info 2023-08-07 14:57:02,455 DEBUG [StoreOpener-e4c09ac0e270308abcead3a999ce0bea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/info 2023-08-07 14:57:02,456 INFO [StoreOpener-e4c09ac0e270308abcead3a999ce0bea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e4c09ac0e270308abcead3a999ce0bea columnFamilyName info 2023-08-07 14:57:02,456 INFO [StoreOpener-e4c09ac0e270308abcead3a999ce0bea-1] regionserver.HStore(310): Store=e4c09ac0e270308abcead3a999ce0bea/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:02,458 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,459 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,463 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:02,467 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:02,468 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e4c09ac0e270308abcead3a999ce0bea; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=741579, jitterRate=-0.057033881545066833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:57:02,468 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e4c09ac0e270308abcead3a999ce0bea: 2023-08-07 14:57:02,470 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea., pid=6, masterSystemTime=1691420222439 2023-08-07 14:57:02,474 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:02,474 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:02,476 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e4c09ac0e270308abcead3a999ce0bea, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:02,476 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420222475"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420222475"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420222475"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420222475"}]},"ts":"1691420222475"} 2023-08-07 14:57:02,483 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-07 14:57:02,483 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure e4c09ac0e270308abcead3a999ce0bea, server=jenkins-hbase4.apache.org,41313,1691420219975 in 194 msec 2023-08-07 14:57:02,487 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-07 14:57:02,487 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e4c09ac0e270308abcead3a999ce0bea, ASSIGN in 358 msec 2023-08-07 14:57:02,489 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 14:57:02,489 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420222489"}]},"ts":"1691420222489"} 2023-08-07 14:57:02,492 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-07 14:57:02,496 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 14:57:02,499 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 524 msec 2023-08-07 14:57:02,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-07 14:57:02,587 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:57:02,587 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:02,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-07 14:57:02,650 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:57:02,656 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 38 msec 2023-08-07 14:57:02,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-07 14:57:02,678 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:57:02,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-08-07 14:57:02,694 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-07 14:57:02,697 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-07 14:57:02,699 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.641sec 2023-08-07 14:57:02,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 14:57:02,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 14:57:02,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 14:57:02,704 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41067,1691420218479-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 14:57:02,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41067,1691420218479-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 14:57:02,715 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 14:57:02,722 DEBUG [Listener at localhost/42441] zookeeper.ReadOnlyZKClient(139): Connect 0x4d22c72d to 127.0.0.1:57140 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:57:02,730 DEBUG [Listener at localhost/42441] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@401682f8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:57:02,773 DEBUG [hconnection-0x287e47ad-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 14:57:02,786 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42014, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 14:57:02,797 INFO [Listener at localhost/42441] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:02,797 INFO [Listener at localhost/42441] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:02,806 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-07 14:57:02,806 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:02,808 INFO [Listener at localhost/42441] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-07 14:57:02,827 WARN [Listener at localhost/42441] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:57:02,829 WARN [Listener at localhost/42441] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:57:02,831 INFO [Listener at localhost/42441] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:57:02,835 INFO [Listener at localhost/42441] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir/Jetty_localhost_33711_datanode____.fbkvvm/webapp 2023-08-07 14:57:02,947 INFO [Listener at localhost/42441] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33711 2023-08-07 14:57:02,960 WARN [Listener at localhost/40997] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:57:02,985 WARN [Listener at localhost/40997] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:57:02,988 WARN [Listener at localhost/40997] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:57:02,989 INFO [Listener at localhost/40997] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:57:02,993 INFO [Listener at localhost/40997] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir/Jetty_localhost_45911_datanode____.uzrpy9/webapp 2023-08-07 14:57:03,096 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf7e1b39939f4a8b8: Processing first storage report for DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8 from datanode 0734b240-1f37-4939-ae0e-0b498b13aba9 2023-08-07 14:57:03,096 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf7e1b39939f4a8b8: from storage DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8 node DatanodeRegistration(127.0.0.1:45235, datanodeUuid=0734b240-1f37-4939-ae0e-0b498b13aba9, infoPort=35277, infoSecurePort=0, ipcPort=40997, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:03,097 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf7e1b39939f4a8b8: Processing first storage report for DS-a10db941-9cd7-4f4f-912a-ca7bf48e85be from datanode 0734b240-1f37-4939-ae0e-0b498b13aba9 2023-08-07 14:57:03,097 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf7e1b39939f4a8b8: from storage DS-a10db941-9cd7-4f4f-912a-ca7bf48e85be node DatanodeRegistration(127.0.0.1:45235, datanodeUuid=0734b240-1f37-4939-ae0e-0b498b13aba9, infoPort=35277, infoSecurePort=0, ipcPort=40997, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:03,107 INFO [Listener at localhost/40997] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45911 2023-08-07 14:57:03,117 WARN [Listener at localhost/41659] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:57:03,151 WARN [Listener at localhost/41659] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:57:03,156 WARN [Listener at localhost/41659] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:57:03,157 INFO [Listener at localhost/41659] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:57:03,167 INFO [Listener at localhost/41659] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir/Jetty_localhost_44529_datanode____.reae71/webapp 2023-08-07 14:57:03,244 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x641bdbe354b8afc8: Processing first storage report for DS-b05beb2c-4fb8-47a9-a174-5324fbb9d98f from datanode b6c75d23-2f88-4249-956d-10fff241442f 2023-08-07 14:57:03,244 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x641bdbe354b8afc8: from storage DS-b05beb2c-4fb8-47a9-a174-5324fbb9d98f node DatanodeRegistration(127.0.0.1:41649, datanodeUuid=b6c75d23-2f88-4249-956d-10fff241442f, infoPort=42179, infoSecurePort=0, ipcPort=41659, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:03,244 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x641bdbe354b8afc8: Processing first storage report for DS-b75fb743-32c0-4eef-90a7-e0dde731dfe6 from datanode b6c75d23-2f88-4249-956d-10fff241442f 2023-08-07 14:57:03,244 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x641bdbe354b8afc8: from storage DS-b75fb743-32c0-4eef-90a7-e0dde731dfe6 node DatanodeRegistration(127.0.0.1:41649, datanodeUuid=b6c75d23-2f88-4249-956d-10fff241442f, infoPort=42179, infoSecurePort=0, ipcPort=41659, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:03,279 INFO [Listener at localhost/41659] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44529 2023-08-07 14:57:03,289 WARN [Listener at localhost/42095] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:57:03,400 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe366d28cfd950bca: Processing first storage report for DS-51f7f720-9fc9-486d-9507-54431d7f4186 from datanode c26b87d9-f669-4b2a-8c30-25f5c1aa67b9 2023-08-07 14:57:03,400 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe366d28cfd950bca: from storage DS-51f7f720-9fc9-486d-9507-54431d7f4186 node DatanodeRegistration(127.0.0.1:36487, datanodeUuid=c26b87d9-f669-4b2a-8c30-25f5c1aa67b9, infoPort=41315, infoSecurePort=0, ipcPort=42095, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-07 14:57:03,401 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe366d28cfd950bca: Processing first storage report for DS-81902d31-c6a2-443f-aebb-86e06f76a0bd from datanode c26b87d9-f669-4b2a-8c30-25f5c1aa67b9 2023-08-07 14:57:03,401 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe366d28cfd950bca: from storage DS-81902d31-c6a2-443f-aebb-86e06f76a0bd node DatanodeRegistration(127.0.0.1:36487, datanodeUuid=c26b87d9-f669-4b2a-8c30-25f5c1aa67b9, infoPort=41315, infoSecurePort=0, ipcPort=42095, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:03,413 DEBUG [Listener at localhost/42095] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-07 14:57:03,416 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47554, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-07 14:57:03,426 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41067] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-07 14:57:03,426 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41067] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-07 14:57:03,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41067] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:03,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41067] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-08-07 14:57:03,436 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 14:57:03,438 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 14:57:03,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41067] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-08-07 14:57:03,442 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,444 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2 empty. 2023-08-07 14:57:03,446 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,446 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-08-07 14:57:03,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41067] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 14:57:03,468 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:03,470 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2573996fc3020b8bb09010c250451ea2, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/.tmp 2023-08-07 14:57:03,491 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:03,491 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing 2573996fc3020b8bb09010c250451ea2, disabling compactions & flushes 2023-08-07 14:57:03,491 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:03,491 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:03,491 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. after waiting 0 ms 2023-08-07 14:57:03,491 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:03,491 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:03,491 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for 2573996fc3020b8bb09010c250451ea2: 2023-08-07 14:57:03,495 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 14:57:03,497 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691420223497"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420223497"}]},"ts":"1691420223497"} 2023-08-07 14:57:03,500 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 14:57:03,502 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 14:57:03,502 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420223502"}]},"ts":"1691420223502"} 2023-08-07 14:57:03,505 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-08-07 14:57:03,509 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=2573996fc3020b8bb09010c250451ea2, ASSIGN}] 2023-08-07 14:57:03,511 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=2573996fc3020b8bb09010c250451ea2, ASSIGN 2023-08-07 14:57:03,513 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=2573996fc3020b8bb09010c250451ea2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41313,1691420219975; forceNewPlan=false, retain=false 2023-08-07 14:57:03,665 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=2573996fc3020b8bb09010c250451ea2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:03,665 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691420223664"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420223664"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420223664"}]},"ts":"1691420223664"} 2023-08-07 14:57:03,668 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 2573996fc3020b8bb09010c250451ea2, server=jenkins-hbase4.apache.org,41313,1691420219975}] 2023-08-07 14:57:03,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:03,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2573996fc3020b8bb09010c250451ea2, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:03,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath 2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:03,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,838 INFO [StoreOpener-2573996fc3020b8bb09010c250451ea2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,841 DEBUG [StoreOpener-2573996fc3020b8bb09010c250451ea2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/info 2023-08-07 14:57:03,841 DEBUG [StoreOpener-2573996fc3020b8bb09010c250451ea2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/info 2023-08-07 14:57:03,841 INFO [StoreOpener-2573996fc3020b8bb09010c250451ea2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2573996fc3020b8bb09010c250451ea2 columnFamilyName info 2023-08-07 14:57:03,842 INFO [StoreOpener-2573996fc3020b8bb09010c250451ea2-1] regionserver.HStore(310): Store=2573996fc3020b8bb09010c250451ea2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:03,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:03,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:03,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2573996fc3020b8bb09010c250451ea2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=864035, jitterRate=0.09867796301841736}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:57:03,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2573996fc3020b8bb09010c250451ea2: 2023-08-07 14:57:03,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2., pid=11, masterSystemTime=1691420223822 2023-08-07 14:57:03,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:03,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:03,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=2573996fc3020b8bb09010c250451ea2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:03,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691420223859"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420223859"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420223859"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420223859"}]},"ts":"1691420223859"} 2023-08-07 14:57:03,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-07 14:57:03,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 2573996fc3020b8bb09010c250451ea2, server=jenkins-hbase4.apache.org,41313,1691420219975 in 194 msec 2023-08-07 14:57:03,869 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-07 14:57:03,870 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=2573996fc3020b8bb09010c250451ea2, ASSIGN in 357 msec 2023-08-07 14:57:03,871 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 14:57:03,871 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420223871"}]},"ts":"1691420223871"} 2023-08-07 14:57:03,873 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-08-07 14:57:03,876 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 14:57:03,878 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 445 msec 2023-08-07 14:57:07,278 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 14:57:07,345 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-07 14:57:07,375 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-07 14:57:07,378 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-08-07 14:57:08,667 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 14:57:08,668 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-07 14:57:13,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41067] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 14:57:13,457 INFO [Listener at localhost/42095] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-08-07 14:57:13,457 DEBUG [Listener at localhost/42095] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-08-07 14:57:13,459 INFO [Listener at localhost/42095] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-07 14:57:13,465 INFO [Listener at localhost/42095] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-08-07 14:57:13,465 INFO [Listener at localhost/42095] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-07 14:57:13,466 INFO [Listener at localhost/42095] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-08-07 14:57:13,477 DEBUG [Listener at localhost/42095] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-08-07 14:57:13,478 DEBUG [Listener at localhost/42095] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:13,479 WARN [Listener at localhost/42095] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:57:13,490 INFO [Listener at localhost/42095] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:57:13,493 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_2004136355_17 at /127.0.0.1:47142 [Receiving block BP-414011224-172.31.14.131-1691420215268:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:39221:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:47142 dst: /127.0.0.1:39221 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,493 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_6960586_17 at /127.0.0.1:47106 [Receiving block BP-414011224-172.31.14.131-1691420215268:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:39221:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:47106 dst: /127.0.0.1:39221 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,494 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_2004136355_17 at /127.0.0.1:42990 [Receiving block BP-414011224-172.31.14.131-1691420215268:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:46283:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:42990 dst: /127.0.0.1:46283 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,494 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:57:13,494 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_6960586_17 at /127.0.0.1:42946 [Receiving block BP-414011224-172.31.14.131-1691420215268:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:46283:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:42946 dst: /127.0.0.1:46283 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,494 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_2004136355_17 at /127.0.0.1:43002 [Receiving block BP-414011224-172.31.14.131-1691420215268:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:46283:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:43002 dst: /127.0.0.1:46283 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,493 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_2004136355_17 at /127.0.0.1:47120 [Receiving block BP-414011224-172.31.14.131-1691420215268:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:39221:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:47120 dst: /127.0.0.1:39221 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,495 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-414011224-172.31.14.131-1691420215268 (Datanode Uuid 63b7ec26-c108-4c34-b46b-bdea9dc84261) service to localhost/127.0.0.1:44501 2023-08-07 14:57:13,498 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data3/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:13,499 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data4/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:13,518 WARN [Listener at localhost/42095] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:57:13,523 WARN [Listener at localhost/42095] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:57:13,524 INFO [Listener at localhost/42095] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:57:13,531 INFO [Listener at localhost/42095] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/java.io.tmpdir/Jetty_localhost_33765_datanode____.5i6seb/webapp 2023-08-07 14:57:13,655 INFO [Listener at localhost/42095] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33765 2023-08-07 14:57:13,674 WARN [Listener at localhost/44177] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:57:13,692 WARN [AsyncFSWAL-0-hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4-prefix:jenkins-hbase4.apache.org,41313,1691420219975] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,694 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C41313%2C1691420219975:(num 1691420221507) roll requested 2023-08-07 14:57:13,722 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45235,DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8,DISK] 2023-08-07 14:57:13,722 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41649,DS-b05beb2c-4fb8-47a9-a174-5324fbb9d98f,DISK] 2023-08-07 14:57:13,733 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 with entries=6, filesize=3.19 KB; new WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420233694 2023-08-07 14:57:13,734 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45235,DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8,DISK], DatanodeInfoWithStorage[127.0.0.1:41649,DS-b05beb2c-4fb8-47a9-a174-5324fbb9d98f,DISK]] 2023-08-07 14:57:13,735 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 is not closed yet, will try archiving it next time 2023-08-07 14:57:13,736 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-07 14:57:13,737 INFO [Listener at localhost/44177] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-07 14:57:13,738 DEBUG [Listener at localhost/44177] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d22c72d to 127.0.0.1:57140 2023-08-07 14:57:13,739 DEBUG [Listener at localhost/44177] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:57:13,739 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,739 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,739 DEBUG [Listener at localhost/44177] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 14:57:13,740 DEBUG [Listener at localhost/44177] util.JVMClusterUtil(257): Found active master hash=1464906003, stopped=false 2023-08-07 14:57:13,740 INFO [Listener at localhost/44177] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:13,742 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 2023-08-07 14:57:13,742 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 14:57:13,742 INFO [Listener at localhost/44177] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 14:57:13,742 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 14:57:13,742 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:13,743 DEBUG [Listener at localhost/44177] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0b154b7e to 127.0.0.1:57140 2023-08-07 14:57:13,743 DEBUG [Listener at localhost/44177] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:57:13,743 INFO [Listener at localhost/44177] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41313,1691420219975' ***** 2023-08-07 14:57:13,743 INFO [Listener at localhost/44177] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 14:57:13,744 INFO [RS:0;jenkins-hbase4:41313] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 14:57:13,745 INFO [RS:0;jenkins-hbase4:41313] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 14:57:13,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:57:13,745 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 14:57:13,745 INFO [RS:0;jenkins-hbase4:41313] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 14:57:13,746 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(3305): Received CLOSE for e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:13,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:57:13,749 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(3305): Received CLOSE for 2573996fc3020b8bb09010c250451ea2 2023-08-07 14:57:13,749 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:13,749 DEBUG [RS:0;jenkins-hbase4:41313] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20097838 to 127.0.0.1:57140 2023-08-07 14:57:13,749 DEBUG [RS:0;jenkins-hbase4:41313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:57:13,749 INFO [RS:0;jenkins-hbase4:41313] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 14:57:13,749 INFO [RS:0;jenkins-hbase4:41313] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 14:57:13,750 INFO [RS:0;jenkins-hbase4:41313] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 14:57:13,750 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 14:57:13,750 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-07 14:57:13,750 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1478): Online Regions={e4c09ac0e270308abcead3a999ce0bea=hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea., 1588230740=hbase:meta,,1.1588230740, 2573996fc3020b8bb09010c250451ea2=TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2.} 2023-08-07 14:57:13,751 WARN [IPC Server handler 3 on default port 44501] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 has not been closed. Lease recovery is in progress. RecoveryId = 1015 for block blk_1073741832_1008 2023-08-07 14:57:13,752 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1504): Waiting on 1588230740, 2573996fc3020b8bb09010c250451ea2, e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:13,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e4c09ac0e270308abcead3a999ce0bea, disabling compactions & flushes 2023-08-07 14:57:13,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:13,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:13,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. after waiting 0 ms 2023-08-07 14:57:13,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:13,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e4c09ac0e270308abcead3a999ce0bea 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-07 14:57:13,754 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 14:57:13,754 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 14:57:13,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 14:57:13,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 14:57:13,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 14:57:13,755 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.92 KB heapSize=5.45 KB 2023-08-07 14:57:13,755 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 after 13ms 2023-08-07 14:57:13,773 WARN [AsyncFSWAL-0-hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4-prefix:jenkins-hbase4.apache.org,41313,1691420219975.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,773 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta:.meta(num 1691420221697) roll requested 2023-08-07 14:57:13,817 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41649,DS-b05beb2c-4fb8-47a9-a174-5324fbb9d98f,DISK] 2023-08-07 14:57:13,817 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45235,DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8,DISK] 2023-08-07 14:57:13,827 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420233774.meta 2023-08-07 14:57:13,832 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45235,DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8,DISK], DatanodeInfoWithStorage[127.0.0.1:41649,DS-b05beb2c-4fb8-47a9-a174-5324fbb9d98f,DISK]] 2023-08-07 14:57:13,832 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta is not closed yet, will try archiving it next time 2023-08-07 14:57:13,832 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,834 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:13,835 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta 2023-08-07 14:57:13,837 WARN [IPC Server handler 0 on default port 44501] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta has not been closed. Lease recovery is in progress. RecoveryId = 1017 for block blk_1073741833_1009 2023-08-07 14:57:13,838 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta after 2ms 2023-08-07 14:57:13,862 WARN [Thread-341] hdfs.DataStreamer(1658): Abandoning BP-414011224-172.31.14.131-1691420215268:blk_1073741840_1018 2023-08-07 14:57:13,866 WARN [Thread-341] hdfs.DataStreamer(1663): Excluding datanode DatanodeInfoWithStorage[127.0.0.1:46283,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK] 2023-08-07 14:57:13,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/.tmp/info/c1c5e1caedfa48ba93179927040a97d7 2023-08-07 14:57:13,919 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x84dd30c438eb637f: Processing first storage report for DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5 from datanode 63b7ec26-c108-4c34-b46b-bdea9dc84261 2023-08-07 14:57:13,926 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x84dd30c438eb637f: from storage DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5 node DatanodeRegistration(127.0.0.1:37291, datanodeUuid=63b7ec26-c108-4c34-b46b-bdea9dc84261, infoPort=45135, infoSecurePort=0, ipcPort=44177, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 5, hasStaleStorage: false, processing time: 7 msecs, invalidatedBlocks: 0 2023-08-07 14:57:13,927 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x84dd30c438eb637f: Processing first storage report for DS-6b6c1f86-3644-45fc-bd90-ac36bbd99007 from datanode 63b7ec26-c108-4c34-b46b-bdea9dc84261 2023-08-07 14:57:13,927 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x84dd30c438eb637f: from storage DS-6b6c1f86-3644-45fc-bd90-ac36bbd99007 node DatanodeRegistration(127.0.0.1:37291, datanodeUuid=63b7ec26-c108-4c34-b46b-bdea9dc84261, infoPort=45135, infoSecurePort=0, ipcPort=44177, storageInfo=lv=-57;cid=testClusterID;nsid=750918687;c=1691420215268), blocks: 6, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:13,930 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/.tmp/info/fce175cbbc99459193ca349fbe10b22b 2023-08-07 14:57:13,953 DEBUG [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1504): Waiting on 1588230740, 2573996fc3020b8bb09010c250451ea2, e4c09ac0e270308abcead3a999ce0bea 2023-08-07 14:57:13,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/.tmp/info/c1c5e1caedfa48ba93179927040a97d7 as hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/info/c1c5e1caedfa48ba93179927040a97d7 2023-08-07 14:57:13,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/info/c1c5e1caedfa48ba93179927040a97d7, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 14:57:13,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for e4c09ac0e270308abcead3a999ce0bea in 231ms, sequenceid=6, compaction requested=false 2023-08-07 14:57:14,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/namespace/e4c09ac0e270308abcead3a999ce0bea/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 14:57:14,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:14,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e4c09ac0e270308abcead3a999ce0bea: 2023-08-07 14:57:14,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691420221972.e4c09ac0e270308abcead3a999ce0bea. 2023-08-07 14:57:14,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2573996fc3020b8bb09010c250451ea2, disabling compactions & flushes 2023-08-07 14:57:14,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:14,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:14,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. after waiting 0 ms 2023-08-07 14:57:14,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:14,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2573996fc3020b8bb09010c250451ea2 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-08-07 14:57:14,022 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/.tmp/table/125b5d23851445c9b901a21c33a7b0ea 2023-08-07 14:57:14,042 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/.tmp/info/fce175cbbc99459193ca349fbe10b22b as hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/info/fce175cbbc99459193ca349fbe10b22b 2023-08-07 14:57:14,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/.tmp/info/910e409dfbdb42d6894d6207ebca76de 2023-08-07 14:57:14,058 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/info/fce175cbbc99459193ca349fbe10b22b, entries=20, sequenceid=14, filesize=7.4 K 2023-08-07 14:57:14,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/.tmp/info/910e409dfbdb42d6894d6207ebca76de as hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/info/910e409dfbdb42d6894d6207ebca76de 2023-08-07 14:57:14,067 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/.tmp/table/125b5d23851445c9b901a21c33a7b0ea as hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/table/125b5d23851445c9b901a21c33a7b0ea 2023-08-07 14:57:14,076 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/info/910e409dfbdb42d6894d6207ebca76de, entries=2, sequenceid=6, filesize=6.8 K 2023-08-07 14:57:14,081 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/table/125b5d23851445c9b901a21c33a7b0ea, entries=4, sequenceid=14, filesize=4.8 K 2023-08-07 14:57:14,083 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for 2573996fc3020b8bb09010c250451ea2 in 67ms, sequenceid=6, compaction requested=false 2023-08-07 14:57:14,084 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.92 KB/2994, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 329ms, sequenceid=14, compaction requested=false 2023-08-07 14:57:14,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/default/TestLogRolling-testLogRollOnDatanodeDeath/2573996fc3020b8bb09010c250451ea2/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 14:57:14,109 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:14,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2573996fc3020b8bb09010c250451ea2: 2023-08-07 14:57:14,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1691420223426.2573996fc3020b8bb09010c250451ea2. 2023-08-07 14:57:14,118 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-07 14:57:14,119 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 14:57:14,121 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 14:57:14,121 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 14:57:14,121 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 14:57:14,153 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41313,1691420219975; all regions closed. 2023-08-07 14:57:14,356 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 14:57:14,356 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 14:57:15,373 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 14:57:17,756 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 after 4014ms 2023-08-07 14:57:17,757 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/oldWALs/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420221507 2023-08-07 14:57:17,839 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta after 4003ms 2023-08-07 14:57:17,840 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/oldWALs/jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta.1691420221697.meta 2023-08-07 14:57:17,855 DEBUG [RS:0;jenkins-hbase4:41313] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/oldWALs 2023-08-07 14:57:17,855 INFO [RS:0;jenkins-hbase4:41313] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41313%2C1691420219975.meta:.meta(num 1691420233774) 2023-08-07 14:57:17,863 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/WALs/jenkins-hbase4.apache.org,41313,1691420219975/jenkins-hbase4.apache.org%2C41313%2C1691420219975.1691420233694 not finished, retry = 0 2023-08-07 14:57:17,974 DEBUG [RS:0;jenkins-hbase4:41313] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/oldWALs 2023-08-07 14:57:17,974 INFO [RS:0;jenkins-hbase4:41313] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41313%2C1691420219975:(num 1691420233694) 2023-08-07 14:57:17,974 DEBUG [RS:0;jenkins-hbase4:41313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:57:17,974 INFO [RS:0;jenkins-hbase4:41313] regionserver.LeaseManager(133): Closed leases 2023-08-07 14:57:17,975 INFO [RS:0;jenkins-hbase4:41313] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 14:57:17,975 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 14:57:17,976 INFO [RS:0;jenkins-hbase4:41313] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41313 2023-08-07 14:57:17,985 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 14:57:17,985 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41313,1691420219975 2023-08-07 14:57:17,985 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 14:57:17,986 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41313,1691420219975] 2023-08-07 14:57:17,986 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41313,1691420219975; numProcessing=1 2023-08-07 14:57:17,995 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41313,1691420219975 already deleted, retry=false 2023-08-07 14:57:17,995 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41313,1691420219975 expired; onlineServers=0 2023-08-07 14:57:17,995 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41067,1691420218479' ***** 2023-08-07 14:57:17,995 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 14:57:17,996 DEBUG [M:0;jenkins-hbase4:41067] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@166dbb28, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 14:57:17,996 INFO [M:0;jenkins-hbase4:41067] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41067,1691420218479 2023-08-07 14:57:17,996 INFO [M:0;jenkins-hbase4:41067] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41067,1691420218479; all regions closed. 2023-08-07 14:57:17,996 DEBUG [M:0;jenkins-hbase4:41067] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:57:17,996 DEBUG [M:0;jenkins-hbase4:41067] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 14:57:17,996 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 14:57:17,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420221200] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420221200,5,FailOnTimeoutGroup] 2023-08-07 14:57:17,996 DEBUG [M:0;jenkins-hbase4:41067] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 14:57:17,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420221199] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420221199,5,FailOnTimeoutGroup] 2023-08-07 14:57:18,006 INFO [M:0;jenkins-hbase4:41067] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 14:57:18,007 INFO [M:0;jenkins-hbase4:41067] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 14:57:17,999 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-07 14:57:18,008 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:18,008 INFO [M:0;jenkins-hbase4:41067] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 14:57:18,009 DEBUG [M:0;jenkins-hbase4:41067] master.HMaster(1512): Stopping service threads 2023-08-07 14:57:18,009 INFO [M:0;jenkins-hbase4:41067] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 14:57:18,009 ERROR [M:0;jenkins-hbase4:41067] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-08-07 14:57:18,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:57:18,010 INFO [M:0;jenkins-hbase4:41067] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 14:57:18,010 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 14:57:18,011 DEBUG [M:0;jenkins-hbase4:41067] zookeeper.ZKUtil(398): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-07 14:57:18,011 WARN [M:0;jenkins-hbase4:41067] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 14:57:18,011 INFO [M:0;jenkins-hbase4:41067] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 14:57:18,012 INFO [M:0;jenkins-hbase4:41067] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 14:57:18,012 DEBUG [M:0;jenkins-hbase4:41067] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 14:57:18,012 INFO [M:0;jenkins-hbase4:41067] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:18,013 DEBUG [M:0;jenkins-hbase4:41067] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:18,013 DEBUG [M:0;jenkins-hbase4:41067] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 14:57:18,013 DEBUG [M:0;jenkins-hbase4:41067] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:18,013 INFO [M:0;jenkins-hbase4:41067] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.07 KB heapSize=45.73 KB 2023-08-07 14:57:18,025 WARN [AsyncFSWAL-0-hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData-prefix:jenkins-hbase4.apache.org,41067,1691420218479] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:18,025 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C41067%2C1691420218479:(num 1691420220509) roll requested 2023-08-07 14:57:18,051 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37291,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK] 2023-08-07 14:57:18,052 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45235,DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8,DISK] 2023-08-07 14:57:18,059 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 with entries=89, filesize=43.92 KB; new WAL /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420238026 2023-08-07 14:57:18,065 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37291,DS-837077d2-0fbf-45b1-855c-ea1e3a03c1b5,DISK], DatanodeInfoWithStorage[127.0.0.1:45235,DS-98449290-1e2d-4235-b7dc-d4cdec3dccd8,DISK]] 2023-08-07 14:57:18,065 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:18,065 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-07 14:57:18,065 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 is not closed yet, will try archiving it next time 2023-08-07 14:57:18,065 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 2023-08-07 14:57:18,067 WARN [IPC Server handler 2 on default port 44501] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 has not been closed. Lease recovery is in progress. RecoveryId = 1024 for block blk_1073741829_1005 2023-08-07 14:57:18,068 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 after 3ms 2023-08-07 14:57:18,094 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:57:18,094 INFO [RS:0;jenkins-hbase4:41313] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41313,1691420219975; zookeeper connection closed. 2023-08-07 14:57:18,094 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): regionserver:41313-0x101dfeed3ce0001, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:57:18,111 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3985f889] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3985f889 2023-08-07 14:57:18,114 INFO [Listener at localhost/44177] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-07 14:57:18,138 INFO [M:0;jenkins-hbase4:41067] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.07 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9f9fed919bfe4e229a0360dd3cc9b50e 2023-08-07 14:57:18,157 DEBUG [M:0;jenkins-hbase4:41067] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9f9fed919bfe4e229a0360dd3cc9b50e as hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9f9fed919bfe4e229a0360dd3cc9b50e 2023-08-07 14:57:18,173 INFO [M:0;jenkins-hbase4:41067] regionserver.HStore(1080): Added hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9f9fed919bfe4e229a0360dd3cc9b50e, entries=11, sequenceid=91, filesize=7.0 K 2023-08-07 14:57:18,175 INFO [M:0;jenkins-hbase4:41067] regionserver.HRegion(2948): Finished flush of dataSize ~38.07 KB/38985, heapSize ~45.72 KB/46816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 162ms, sequenceid=91, compaction requested=false 2023-08-07 14:57:18,186 INFO [M:0;jenkins-hbase4:41067] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:18,186 DEBUG [M:0;jenkins-hbase4:41067] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:57:18,666 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 14:57:19,568 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 14:57:22,069 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 after 4004ms 2023-08-07 14:57:22,070 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/WALs/jenkins-hbase4.apache.org,41067,1691420218479/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/oldWALs/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 2023-08-07 14:57:22,077 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/MasterData/oldWALs/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509 to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/oldWALs/jenkins-hbase4.apache.org%2C41067%2C1691420218479.1691420220509$masterlocalwal$ 2023-08-07 14:57:22,077 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 14:57:22,077 INFO [M:0;jenkins-hbase4:41067] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 14:57:22,078 INFO [M:0;jenkins-hbase4:41067] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41067 2023-08-07 14:57:22,081 DEBUG [M:0;jenkins-hbase4:41067] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,41067,1691420218479 already deleted, retry=false 2023-08-07 14:57:22,183 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:57:22,183 INFO [M:0;jenkins-hbase4:41067] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41067,1691420218479; zookeeper connection closed. 2023-08-07 14:57:22,183 DEBUG [Listener at localhost/42441-EventThread] zookeeper.ZKWatcher(600): master:41067-0x101dfeed3ce0000, quorum=127.0.0.1:57140, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:57:22,185 WARN [Listener at localhost/44177] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:57:22,191 INFO [Listener at localhost/44177] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:57:22,296 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:57:22,296 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-414011224-172.31.14.131-1691420215268 (Datanode Uuid 63b7ec26-c108-4c34-b46b-bdea9dc84261) service to localhost/127.0.0.1:44501 2023-08-07 14:57:22,297 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data3/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,297 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data4/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,301 WARN [Listener at localhost/44177] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:57:22,304 INFO [Listener at localhost/44177] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:57:22,407 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:57:22,407 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-414011224-172.31.14.131-1691420215268 (Datanode Uuid c26b87d9-f669-4b2a-8c30-25f5c1aa67b9) service to localhost/127.0.0.1:44501 2023-08-07 14:57:22,408 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data9/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,408 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data10/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,410 WARN [Listener at localhost/44177] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:57:22,417 INFO [Listener at localhost/44177] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:57:22,521 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:57:22,521 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-414011224-172.31.14.131-1691420215268 (Datanode Uuid b6c75d23-2f88-4249-956d-10fff241442f) service to localhost/127.0.0.1:44501 2023-08-07 14:57:22,522 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data7/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,522 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data8/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,523 WARN [Listener at localhost/44177] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:57:22,528 INFO [Listener at localhost/44177] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:57:22,631 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:57:22,631 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-414011224-172.31.14.131-1691420215268 (Datanode Uuid 0734b240-1f37-4939-ae0e-0b498b13aba9) service to localhost/127.0.0.1:44501 2023-08-07 14:57:22,632 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data5/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,632 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data6/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,633 WARN [Listener at localhost/44177] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:57:22,639 INFO [Listener at localhost/44177] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:57:22,741 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:57:22,741 WARN [BP-414011224-172.31.14.131-1691420215268 heartbeating to localhost/127.0.0.1:44501] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-414011224-172.31.14.131-1691420215268 (Datanode Uuid 8beb4d85-3072-419e-b9cc-cf1c03909e14) service to localhost/127.0.0.1:44501 2023-08-07 14:57:22,741 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data1/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,742 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/cluster_3bd2e09c-f200-57aa-b12c-18a069abe25d/dfs/data/data2/current/BP-414011224-172.31.14.131-1691420215268] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:57:22,780 INFO [Listener at localhost/44177] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:57:22,896 INFO [Listener at localhost/44177] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-07 14:57:22,945 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-07 14:57:22,957 INFO [Listener at localhost/44177] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (2131506352) connection to localhost/127.0.0.1:44501 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/44177 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: IPC Client (2131506352) connection to localhost/127.0.0.1:44501 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (2131506352) connection to localhost/127.0.0.1:44501 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@3d86f0a5 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:44501 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:44501 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=469 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=387 (was 450), ProcessCount=171 (was 171), AvailableMemoryMB=1935 (was 1942) 2023-08-07 14:57:22,966 INFO [Listener at localhost/44177] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=469, MaxFileDescriptor=60000, SystemLoadAverage=387, ProcessCount=171, AvailableMemoryMB=1935 2023-08-07 14:57:22,966 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-07 14:57:22,967 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/hadoop.log.dir so I do NOT create it in target/test-data/ada65793-2523-41b4-2048-16601238cf5b 2023-08-07 14:57:22,967 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f738007-85a1-fb36-874d-71f515dbf8ac/hadoop.tmp.dir so I do NOT create it in target/test-data/ada65793-2523-41b4-2048-16601238cf5b 2023-08-07 14:57:22,967 INFO [Listener at localhost/44177] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684, deleteOnExit=true 2023-08-07 14:57:22,967 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-07 14:57:22,967 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/test.cache.data in system properties and HBase conf 2023-08-07 14:57:22,968 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/hadoop.tmp.dir in system properties and HBase conf 2023-08-07 14:57:22,968 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/hadoop.log.dir in system properties and HBase conf 2023-08-07 14:57:22,968 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-07 14:57:22,968 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-07 14:57:22,968 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-07 14:57:22,969 DEBUG [Listener at localhost/44177] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-07 14:57:22,969 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-07 14:57:22,969 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-07 14:57:22,970 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-07 14:57:22,970 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 14:57:22,970 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-07 14:57:22,970 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-07 14:57:22,970 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 14:57:22,970 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 14:57:22,970 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-07 14:57:22,971 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/nfs.dump.dir in system properties and HBase conf 2023-08-07 14:57:22,971 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/java.io.tmpdir in system properties and HBase conf 2023-08-07 14:57:22,971 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 14:57:22,971 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-07 14:57:22,971 INFO [Listener at localhost/44177] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-07 14:57:22,975 WARN [Listener at localhost/44177] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 14:57:22,975 WARN [Listener at localhost/44177] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 14:57:23,026 WARN [Listener at localhost/44177] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:57:23,029 INFO [Listener at localhost/44177] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:57:23,036 INFO [Listener at localhost/44177] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/java.io.tmpdir/Jetty_localhost_44797_hdfs____wr9cqn/webapp 2023-08-07 14:57:23,127 INFO [Listener at localhost/44177] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44797 2023-08-07 14:57:23,132 WARN [Listener at localhost/44177] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 14:57:23,132 WARN [Listener at localhost/44177] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 14:57:23,174 WARN [Listener at localhost/44739] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:57:23,191 WARN [Listener at localhost/44739] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:57:23,194 WARN [Listener at localhost/44739] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:57:23,196 INFO [Listener at localhost/44739] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:57:23,201 INFO [Listener at localhost/44739] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/java.io.tmpdir/Jetty_localhost_39375_datanode____.m6xxta/webapp 2023-08-07 14:57:23,293 INFO [Listener at localhost/44739] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39375 2023-08-07 14:57:23,299 WARN [Listener at localhost/44843] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:57:23,315 WARN [Listener at localhost/44843] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:57:23,317 WARN [Listener at localhost/44843] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:57:23,319 INFO [Listener at localhost/44843] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:57:23,322 INFO [Listener at localhost/44843] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/java.io.tmpdir/Jetty_localhost_40893_datanode____.j9rflv/webapp 2023-08-07 14:57:23,411 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfda2c80f4f9d7bdf: Processing first storage report for DS-60d74a13-8a3e-4572-b363-abaab0da7cef from datanode 735f46f1-9a6a-436b-8f24-8e9a9ad24cde 2023-08-07 14:57:23,411 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfda2c80f4f9d7bdf: from storage DS-60d74a13-8a3e-4572-b363-abaab0da7cef node DatanodeRegistration(127.0.0.1:33501, datanodeUuid=735f46f1-9a6a-436b-8f24-8e9a9ad24cde, infoPort=39065, infoSecurePort=0, ipcPort=44843, storageInfo=lv=-57;cid=testClusterID;nsid=66930525;c=1691420242978), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-07 14:57:23,412 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfda2c80f4f9d7bdf: Processing first storage report for DS-557a4dfd-956f-49a1-81b4-580a02fb5985 from datanode 735f46f1-9a6a-436b-8f24-8e9a9ad24cde 2023-08-07 14:57:23,412 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfda2c80f4f9d7bdf: from storage DS-557a4dfd-956f-49a1-81b4-580a02fb5985 node DatanodeRegistration(127.0.0.1:33501, datanodeUuid=735f46f1-9a6a-436b-8f24-8e9a9ad24cde, infoPort=39065, infoSecurePort=0, ipcPort=44843, storageInfo=lv=-57;cid=testClusterID;nsid=66930525;c=1691420242978), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:23,426 INFO [Listener at localhost/44843] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40893 2023-08-07 14:57:23,438 WARN [Listener at localhost/41565] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:57:23,544 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa5e51e796c0840a4: Processing first storage report for DS-8dc195d8-0fe9-4115-94b2-e64509866246 from datanode d228b469-a6c9-42a4-8497-5a87e67e31f7 2023-08-07 14:57:23,544 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa5e51e796c0840a4: from storage DS-8dc195d8-0fe9-4115-94b2-e64509866246 node DatanodeRegistration(127.0.0.1:45833, datanodeUuid=d228b469-a6c9-42a4-8497-5a87e67e31f7, infoPort=37087, infoSecurePort=0, ipcPort=41565, storageInfo=lv=-57;cid=testClusterID;nsid=66930525;c=1691420242978), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:57:23,544 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa5e51e796c0840a4: Processing first storage report for DS-ec0534b5-00c6-4daf-90ec-75bb884ded41 from datanode d228b469-a6c9-42a4-8497-5a87e67e31f7 2023-08-07 14:57:23,544 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa5e51e796c0840a4: from storage DS-ec0534b5-00c6-4daf-90ec-75bb884ded41 node DatanodeRegistration(127.0.0.1:45833, datanodeUuid=d228b469-a6c9-42a4-8497-5a87e67e31f7, infoPort=37087, infoSecurePort=0, ipcPort=41565, storageInfo=lv=-57;cid=testClusterID;nsid=66930525;c=1691420242978), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-07 14:57:23,550 DEBUG [Listener at localhost/41565] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b 2023-08-07 14:57:23,557 INFO [Listener at localhost/41565] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684/zookeeper_0, clientPort=55126, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-07 14:57:23,559 INFO [Listener at localhost/41565] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55126 2023-08-07 14:57:23,559 INFO [Listener at localhost/41565] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:23,560 INFO [Listener at localhost/41565] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:23,581 INFO [Listener at localhost/41565] util.FSUtils(471): Created version file at hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082 with version=8 2023-08-07 14:57:23,581 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/hbase-staging 2023-08-07 14:57:23,583 INFO [Listener at localhost/41565] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 14:57:23,583 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:57:23,583 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 14:57:23,584 INFO [Listener at localhost/41565] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 14:57:23,584 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:57:23,584 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 14:57:23,584 INFO [Listener at localhost/41565] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 14:57:23,585 INFO [Listener at localhost/41565] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36933 2023-08-07 14:57:23,585 INFO [Listener at localhost/41565] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:23,587 INFO [Listener at localhost/41565] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:23,588 INFO [Listener at localhost/41565] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36933 connecting to ZooKeeper ensemble=127.0.0.1:55126 2023-08-07 14:57:23,594 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:369330x0, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 14:57:23,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:36933-0x101dfef39670000 connected 2023-08-07 14:57:23,611 DEBUG [Listener at localhost/41565] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:57:23,612 DEBUG [Listener at localhost/41565] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:57:23,612 DEBUG [Listener at localhost/41565] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 14:57:23,613 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36933 2023-08-07 14:57:23,613 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36933 2023-08-07 14:57:23,613 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36933 2023-08-07 14:57:23,614 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36933 2023-08-07 14:57:23,614 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36933 2023-08-07 14:57:23,615 INFO [Listener at localhost/41565] master.HMaster(444): hbase.rootdir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082, hbase.cluster.distributed=false 2023-08-07 14:57:23,632 INFO [Listener at localhost/41565] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 14:57:23,632 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:57:23,632 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 14:57:23,632 INFO [Listener at localhost/41565] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 14:57:23,632 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:57:23,632 INFO [Listener at localhost/41565] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 14:57:23,633 INFO [Listener at localhost/41565] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 14:57:23,635 INFO [Listener at localhost/41565] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39873 2023-08-07 14:57:23,635 INFO [Listener at localhost/41565] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 14:57:23,636 DEBUG [Listener at localhost/41565] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 14:57:23,637 INFO [Listener at localhost/41565] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:23,638 INFO [Listener at localhost/41565] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:23,639 INFO [Listener at localhost/41565] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39873 connecting to ZooKeeper ensemble=127.0.0.1:55126 2023-08-07 14:57:23,642 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:398730x0, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 14:57:23,645 DEBUG [Listener at localhost/41565] zookeeper.ZKUtil(164): regionserver:398730x0, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:57:23,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39873-0x101dfef39670001 connected 2023-08-07 14:57:23,646 DEBUG [Listener at localhost/41565] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:57:23,646 DEBUG [Listener at localhost/41565] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 14:57:23,647 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39873 2023-08-07 14:57:23,651 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39873 2023-08-07 14:57:23,652 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39873 2023-08-07 14:57:23,654 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39873 2023-08-07 14:57:23,654 DEBUG [Listener at localhost/41565] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39873 2023-08-07 14:57:23,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:23,658 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 14:57:23,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:23,661 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 14:57:23,661 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 14:57:23,661 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:23,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 14:57:23,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 14:57:23,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,36933,1691420243583 from backup master directory 2023-08-07 14:57:23,666 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:23,666 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 14:57:23,666 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 14:57:23,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:23,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/hbase.id with ID: 6b44db04-9efd-49d7-8e4d-57f0dc1cee6c 2023-08-07 14:57:23,707 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:23,709 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:23,724 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7633abe7 to 127.0.0.1:55126 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:57:23,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16c510e9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:57:23,729 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:23,729 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 14:57:23,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:57:23,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store-tmp 2023-08-07 14:57:23,742 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:23,742 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 14:57:23,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:23,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:23,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 14:57:23,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:23,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:57:23,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:57:23,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/WALs/jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:23,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36933%2C1691420243583, suffix=, logDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/WALs/jenkins-hbase4.apache.org,36933,1691420243583, archiveDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/oldWALs, maxLogs=10 2023-08-07 14:57:23,773 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK] 2023-08-07 14:57:23,773 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK] 2023-08-07 14:57:23,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/WALs/jenkins-hbase4.apache.org,36933,1691420243583/jenkins-hbase4.apache.org%2C36933%2C1691420243583.1691420243748 2023-08-07 14:57:23,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK], DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK]] 2023-08-07 14:57:23,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:23,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:23,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:23,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:23,782 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:23,784 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 14:57:23,785 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 14:57:23,786 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:23,787 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:23,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:23,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:57:23,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:23,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=723547, jitterRate=-0.07996293902397156}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:57:23,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:57:23,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 14:57:23,800 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 14:57:23,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 14:57:23,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 14:57:23,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 14:57:23,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-07 14:57:23,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 14:57:23,810 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-07 14:57:23,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-07 14:57:23,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 14:57:23,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 14:57:23,829 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-07 14:57:23,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 14:57:23,830 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-07 14:57:23,836 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:23,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-07 14:57:23,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-07 14:57:23,838 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-07 14:57:23,840 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 14:57:23,840 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 14:57:23,840 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:23,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,36933,1691420243583, sessionid=0x101dfef39670000, setting cluster-up flag (Was=false) 2023-08-07 14:57:23,845 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:23,850 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-07 14:57:23,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:23,855 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:23,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-07 14:57:23,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:23,862 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.hbase-snapshot/.tmp 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 14:57:23,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:23,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691420273875 2023-08-07 14:57:23,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 14:57:23,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 14:57:23,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 14:57:23,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 14:57:23,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 14:57:23,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 14:57:23,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:23,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 14:57:23,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 14:57:23,880 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 14:57:23,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 14:57:23,880 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-07 14:57:23,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 14:57:23,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 14:57:23,881 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:23,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420243880,5,FailOnTimeoutGroup] 2023-08-07 14:57:23,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420243887,5,FailOnTimeoutGroup] 2023-08-07 14:57:23,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:23,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 14:57:23,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:23,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:23,901 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:23,903 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:23,903 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082 2023-08-07 14:57:23,916 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:23,918 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 14:57:23,921 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/info 2023-08-07 14:57:23,922 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 14:57:23,923 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:23,923 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 14:57:23,925 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:57:23,925 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 14:57:23,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:23,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 14:57:23,928 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/table 2023-08-07 14:57:23,929 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 14:57:23,930 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:23,930 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740 2023-08-07 14:57:23,932 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740 2023-08-07 14:57:23,935 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 14:57:23,937 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 14:57:23,940 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:23,941 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=717935, jitterRate=-0.0870995968580246}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 14:57:23,941 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 14:57:23,941 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 14:57:23,941 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 14:57:23,941 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 14:57:23,942 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 14:57:23,942 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 14:57:23,942 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 14:57:23,942 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 14:57:23,944 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 14:57:23,944 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-07 14:57:23,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 14:57:23,946 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 14:57:23,949 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 14:57:23,957 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(951): ClusterId : 6b44db04-9efd-49d7-8e4d-57f0dc1cee6c 2023-08-07 14:57:23,957 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 14:57:23,961 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 14:57:23,961 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 14:57:23,963 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 14:57:23,964 DEBUG [RS:0;jenkins-hbase4:39873] zookeeper.ReadOnlyZKClient(139): Connect 0x6f99a233 to 127.0.0.1:55126 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:57:23,968 DEBUG [RS:0;jenkins-hbase4:39873] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@615e51b0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:57:23,969 DEBUG [RS:0;jenkins-hbase4:39873] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6561c943, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 14:57:23,982 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39873 2023-08-07 14:57:23,982 INFO [RS:0;jenkins-hbase4:39873] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 14:57:23,982 INFO [RS:0;jenkins-hbase4:39873] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 14:57:23,983 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 14:57:23,983 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36933,1691420243583 with isa=jenkins-hbase4.apache.org/172.31.14.131:39873, startcode=1691420243631 2023-08-07 14:57:23,984 DEBUG [RS:0;jenkins-hbase4:39873] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 14:57:23,987 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39453, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 14:57:23,988 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36933] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:23,989 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082 2023-08-07 14:57:23,989 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:44739 2023-08-07 14:57:23,989 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 14:57:23,991 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 14:57:23,992 DEBUG [RS:0;jenkins-hbase4:39873] zookeeper.ZKUtil(162): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:23,992 WARN [RS:0;jenkins-hbase4:39873] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 14:57:23,992 INFO [RS:0;jenkins-hbase4:39873] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:57:23,992 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1948): logDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:23,997 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39873,1691420243631] 2023-08-07 14:57:24,001 DEBUG [RS:0;jenkins-hbase4:39873] zookeeper.ZKUtil(162): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:24,003 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-07 14:57:24,003 INFO [RS:0;jenkins-hbase4:39873] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 14:57:24,009 INFO [RS:0;jenkins-hbase4:39873] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 14:57:24,013 INFO [RS:0;jenkins-hbase4:39873] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 14:57:24,013 INFO [RS:0;jenkins-hbase4:39873] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,013 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 14:57:24,015 INFO [RS:0;jenkins-hbase4:39873] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,015 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,015 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,015 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,016 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,016 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,016 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 14:57:24,016 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,016 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,016 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,016 DEBUG [RS:0;jenkins-hbase4:39873] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:57:24,017 INFO [RS:0;jenkins-hbase4:39873] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,017 INFO [RS:0;jenkins-hbase4:39873] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,017 INFO [RS:0;jenkins-hbase4:39873] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,030 INFO [RS:0;jenkins-hbase4:39873] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 14:57:24,030 INFO [RS:0;jenkins-hbase4:39873] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39873,1691420243631-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,042 INFO [RS:0;jenkins-hbase4:39873] regionserver.Replication(203): jenkins-hbase4.apache.org,39873,1691420243631 started 2023-08-07 14:57:24,042 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39873,1691420243631, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39873, sessionid=0x101dfef39670001 2023-08-07 14:57:24,042 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 14:57:24,042 DEBUG [RS:0;jenkins-hbase4:39873] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:24,042 DEBUG [RS:0;jenkins-hbase4:39873] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39873,1691420243631' 2023-08-07 14:57:24,042 DEBUG [RS:0;jenkins-hbase4:39873] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:57:24,043 DEBUG [RS:0;jenkins-hbase4:39873] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:24,043 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 14:57:24,044 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 14:57:24,044 DEBUG [RS:0;jenkins-hbase4:39873] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:24,044 DEBUG [RS:0;jenkins-hbase4:39873] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39873,1691420243631' 2023-08-07 14:57:24,044 DEBUG [RS:0;jenkins-hbase4:39873] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-07 14:57:24,044 DEBUG [RS:0;jenkins-hbase4:39873] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-07 14:57:24,044 DEBUG [RS:0;jenkins-hbase4:39873] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 14:57:24,045 INFO [RS:0;jenkins-hbase4:39873] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 14:57:24,045 INFO [RS:0;jenkins-hbase4:39873] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 14:57:24,100 DEBUG [jenkins-hbase4:36933] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 14:57:24,101 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39873,1691420243631, state=OPENING 2023-08-07 14:57:24,104 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-07 14:57:24,105 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:24,106 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39873,1691420243631}] 2023-08-07 14:57:24,106 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 14:57:24,148 INFO [RS:0;jenkins-hbase4:39873] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39873%2C1691420243631, suffix=, logDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631, archiveDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/oldWALs, maxLogs=32 2023-08-07 14:57:24,171 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK] 2023-08-07 14:57:24,171 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK] 2023-08-07 14:57:24,174 INFO [RS:0;jenkins-hbase4:39873] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420244150 2023-08-07 14:57:24,174 DEBUG [RS:0;jenkins-hbase4:39873] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK], DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK]] 2023-08-07 14:57:24,261 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:24,262 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 14:57:24,264 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53672, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 14:57:24,269 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 14:57:24,269 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:57:24,272 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39873%2C1691420243631.meta, suffix=.meta, logDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631, archiveDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/oldWALs, maxLogs=32 2023-08-07 14:57:24,291 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK] 2023-08-07 14:57:24,291 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK] 2023-08-07 14:57:24,294 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.meta.1691420244274.meta 2023-08-07 14:57:24,296 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK], DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK]] 2023-08-07 14:57:24,296 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:24,296 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 14:57:24,297 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 14:57:24,298 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 14:57:24,298 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 14:57:24,298 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:24,298 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 14:57:24,298 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 14:57:24,300 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 14:57:24,301 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/info 2023-08-07 14:57:24,301 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/info 2023-08-07 14:57:24,302 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 14:57:24,303 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:24,303 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 14:57:24,305 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:57:24,305 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:57:24,306 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 14:57:24,306 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:24,307 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 14:57:24,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/table 2023-08-07 14:57:24,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/table 2023-08-07 14:57:24,309 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 14:57:24,310 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:24,311 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740 2023-08-07 14:57:24,313 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740 2023-08-07 14:57:24,316 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 14:57:24,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 14:57:24,318 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=767627, jitterRate=-0.023912161588668823}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 14:57:24,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 14:57:24,321 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691420244261 2023-08-07 14:57:24,326 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 14:57:24,326 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 14:57:24,327 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39873,1691420243631, state=OPEN 2023-08-07 14:57:24,329 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-07 14:57:24,329 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 14:57:24,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-07 14:57:24,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39873,1691420243631 in 223 msec 2023-08-07 14:57:24,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-07 14:57:24,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 388 msec 2023-08-07 14:57:24,340 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 469 msec 2023-08-07 14:57:24,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691420244341, completionTime=-1 2023-08-07 14:57:24,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-07 14:57:24,341 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 14:57:24,345 DEBUG [hconnection-0x68d31f38-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 14:57:24,347 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53684, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 14:57:24,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-07 14:57:24,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691420304349 2023-08-07 14:57:24,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691420364349 2023-08-07 14:57:24,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 8 msec 2023-08-07 14:57:24,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36933,1691420243583-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36933,1691420243583-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36933,1691420243583-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:36933, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 14:57:24,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-07 14:57:24,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:24,360 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-07 14:57:24,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-07 14:57:24,362 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 14:57:24,363 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 14:57:24,365 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,366 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f empty. 2023-08-07 14:57:24,367 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,367 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-07 14:57:24,391 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:24,393 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 089f266150a49d62d6fb34b3c6efa48f, NAME => 'hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp 2023-08-07 14:57:24,405 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:24,405 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 089f266150a49d62d6fb34b3c6efa48f, disabling compactions & flushes 2023-08-07 14:57:24,405 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:24,405 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:24,405 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. after waiting 0 ms 2023-08-07 14:57:24,405 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:24,405 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:24,405 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 089f266150a49d62d6fb34b3c6efa48f: 2023-08-07 14:57:24,409 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 14:57:24,412 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420244411"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420244411"}]},"ts":"1691420244411"} 2023-08-07 14:57:24,415 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 14:57:24,416 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 14:57:24,417 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420244416"}]},"ts":"1691420244416"} 2023-08-07 14:57:24,420 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-07 14:57:24,426 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=089f266150a49d62d6fb34b3c6efa48f, ASSIGN}] 2023-08-07 14:57:24,429 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=089f266150a49d62d6fb34b3c6efa48f, ASSIGN 2023-08-07 14:57:24,430 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=089f266150a49d62d6fb34b3c6efa48f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39873,1691420243631; forceNewPlan=false, retain=false 2023-08-07 14:57:24,581 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=089f266150a49d62d6fb34b3c6efa48f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:24,581 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420244581"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420244581"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420244581"}]},"ts":"1691420244581"} 2023-08-07 14:57:24,585 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 089f266150a49d62d6fb34b3c6efa48f, server=jenkins-hbase4.apache.org,39873,1691420243631}] 2023-08-07 14:57:24,743 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:24,744 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 089f266150a49d62d6fb34b3c6efa48f, NAME => 'hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:24,744 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,744 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:24,744 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,744 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,746 INFO [StoreOpener-089f266150a49d62d6fb34b3c6efa48f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,747 DEBUG [StoreOpener-089f266150a49d62d6fb34b3c6efa48f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/info 2023-08-07 14:57:24,747 DEBUG [StoreOpener-089f266150a49d62d6fb34b3c6efa48f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/info 2023-08-07 14:57:24,748 INFO [StoreOpener-089f266150a49d62d6fb34b3c6efa48f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 089f266150a49d62d6fb34b3c6efa48f columnFamilyName info 2023-08-07 14:57:24,748 INFO [StoreOpener-089f266150a49d62d6fb34b3c6efa48f-1] regionserver.HStore(310): Store=089f266150a49d62d6fb34b3c6efa48f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:24,749 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,750 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,753 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:57:24,755 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:24,756 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 089f266150a49d62d6fb34b3c6efa48f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=707096, jitterRate=-0.100881427526474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:57:24,756 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 089f266150a49d62d6fb34b3c6efa48f: 2023-08-07 14:57:24,757 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f., pid=6, masterSystemTime=1691420244738 2023-08-07 14:57:24,760 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:24,760 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:24,761 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=089f266150a49d62d6fb34b3c6efa48f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:24,761 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420244760"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420244760"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420244760"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420244760"}]},"ts":"1691420244760"} 2023-08-07 14:57:24,765 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-07 14:57:24,765 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 089f266150a49d62d6fb34b3c6efa48f, server=jenkins-hbase4.apache.org,39873,1691420243631 in 178 msec 2023-08-07 14:57:24,768 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-07 14:57:24,769 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=089f266150a49d62d6fb34b3c6efa48f, ASSIGN in 339 msec 2023-08-07 14:57:24,770 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 14:57:24,770 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420244770"}]},"ts":"1691420244770"} 2023-08-07 14:57:24,772 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-07 14:57:24,774 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 14:57:24,776 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 416 msec 2023-08-07 14:57:24,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-07 14:57:24,866 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:57:24,866 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:24,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-07 14:57:24,882 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:57:24,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 13 msec 2023-08-07 14:57:24,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-07 14:57:24,901 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:57:24,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 12 msec 2023-08-07 14:57:24,918 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-07 14:57:24,921 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-07 14:57:24,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.254sec 2023-08-07 14:57:24,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 14:57:24,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 14:57:24,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 14:57:24,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36933,1691420243583-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 14:57:24,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36933,1691420243583-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 14:57:24,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 14:57:24,957 DEBUG [Listener at localhost/41565] zookeeper.ReadOnlyZKClient(139): Connect 0x0cf46e7e to 127.0.0.1:55126 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:57:24,962 DEBUG [Listener at localhost/41565] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@421c2fa5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:57:24,963 DEBUG [hconnection-0x82b8dc7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 14:57:24,965 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 14:57:24,967 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:57:24,968 INFO [Listener at localhost/41565] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:57:24,971 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-07 14:57:24,971 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:57:24,972 INFO [Listener at localhost/41565] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-07 14:57:24,974 DEBUG [Listener at localhost/41565] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-07 14:57:24,976 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42482, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-07 14:57:24,978 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-07 14:57:24,978 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-07 14:57:24,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 14:57:24,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:24,982 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 14:57:24,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-08-07 14:57:24,983 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 14:57:24,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 14:57:24,986 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2 2023-08-07 14:57:24,986 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2 empty. 2023-08-07 14:57:24,987 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2 2023-08-07 14:57:24,987 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-08-07 14:57:25,001 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-08-07 14:57:25,002 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => f6063f05f0012476bb995204514580c2, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/.tmp 2023-08-07 14:57:25,014 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:25,014 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing f6063f05f0012476bb995204514580c2, disabling compactions & flushes 2023-08-07 14:57:25,014 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:25,014 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:25,014 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. after waiting 0 ms 2023-08-07 14:57:25,014 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:25,014 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:25,014 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:57:25,017 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 14:57:25,019 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1691420245019"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420245019"}]},"ts":"1691420245019"} 2023-08-07 14:57:25,021 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 14:57:25,022 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 14:57:25,022 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420245022"}]},"ts":"1691420245022"} 2023-08-07 14:57:25,024 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-08-07 14:57:25,028 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=f6063f05f0012476bb995204514580c2, ASSIGN}] 2023-08-07 14:57:25,030 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=f6063f05f0012476bb995204514580c2, ASSIGN 2023-08-07 14:57:25,031 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=f6063f05f0012476bb995204514580c2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39873,1691420243631; forceNewPlan=false, retain=false 2023-08-07 14:57:25,182 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f6063f05f0012476bb995204514580c2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:25,182 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1691420245182"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420245182"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420245182"}]},"ts":"1691420245182"} 2023-08-07 14:57:25,185 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure f6063f05f0012476bb995204514580c2, server=jenkins-hbase4.apache.org,39873,1691420243631}] 2023-08-07 14:57:25,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:25,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f6063f05f0012476bb995204514580c2, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:57:25,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling f6063f05f0012476bb995204514580c2 2023-08-07 14:57:25,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:57:25,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f6063f05f0012476bb995204514580c2 2023-08-07 14:57:25,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f6063f05f0012476bb995204514580c2 2023-08-07 14:57:25,345 INFO [StoreOpener-f6063f05f0012476bb995204514580c2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f6063f05f0012476bb995204514580c2 2023-08-07 14:57:25,346 DEBUG [StoreOpener-f6063f05f0012476bb995204514580c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info 2023-08-07 14:57:25,347 DEBUG [StoreOpener-f6063f05f0012476bb995204514580c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info 2023-08-07 14:57:25,347 INFO [StoreOpener-f6063f05f0012476bb995204514580c2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f6063f05f0012476bb995204514580c2 columnFamilyName info 2023-08-07 14:57:25,348 INFO [StoreOpener-f6063f05f0012476bb995204514580c2-1] regionserver.HStore(310): Store=f6063f05f0012476bb995204514580c2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:57:25,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2 2023-08-07 14:57:25,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2 2023-08-07 14:57:25,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f6063f05f0012476bb995204514580c2 2023-08-07 14:57:25,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:57:25,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f6063f05f0012476bb995204514580c2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=800633, jitterRate=0.018057554960250854}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:57:25,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:57:25,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2., pid=11, masterSystemTime=1691420245338 2023-08-07 14:57:25,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:25,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:25,360 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f6063f05f0012476bb995204514580c2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:25,360 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1691420245359"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420245359"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420245359"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420245359"}]},"ts":"1691420245359"} 2023-08-07 14:57:25,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-07 14:57:25,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure f6063f05f0012476bb995204514580c2, server=jenkins-hbase4.apache.org,39873,1691420243631 in 177 msec 2023-08-07 14:57:25,367 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-07 14:57:25,367 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=f6063f05f0012476bb995204514580c2, ASSIGN in 336 msec 2023-08-07 14:57:25,368 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 14:57:25,368 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420245368"}]},"ts":"1691420245368"} 2023-08-07 14:57:25,370 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-08-07 14:57:25,372 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 14:57:25,374 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 393 msec 2023-08-07 14:57:28,666 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 14:57:28,667 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-07 14:57:29,800 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 14:57:30,003 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-07 14:57:30,004 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-07 14:57:30,004 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:57:34,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 14:57:34,986 INFO [Listener at localhost/41565] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-08-07 14:57:34,989 DEBUG [Listener at localhost/41565] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:34,989 DEBUG [Listener at localhost/41565] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:35,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 14:57:35,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-08-07 14:57:35,009 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-08-07 14:57:35,009 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:57:35,009 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-08-07 14:57:35,009 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-08-07 14:57:35,010 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,010 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-07 14:57:35,011 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:57:35,011 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:57:35,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:35,012 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,012 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 14:57:35,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-08-07 14:57:35,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 14:57:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-07 14:57:35,013 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-08-07 14:57:35,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-08-07 14:57:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-08-07 14:57:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:57:35,016 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-08-07 14:57:35,016 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 14:57:35,016 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 14:57:35,016 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:35,016 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. started... 2023-08-07 14:57:35,017 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 089f266150a49d62d6fb34b3c6efa48f 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-07 14:57:35,029 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/.tmp/info/47ae5917ef964e42b2c710ab7fb52dc9 2023-08-07 14:57:35,039 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/.tmp/info/47ae5917ef964e42b2c710ab7fb52dc9 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/info/47ae5917ef964e42b2c710ab7fb52dc9 2023-08-07 14:57:35,046 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/info/47ae5917ef964e42b2c710ab7fb52dc9, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 14:57:35,048 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 089f266150a49d62d6fb34b3c6efa48f in 31ms, sequenceid=6, compaction requested=false 2023-08-07 14:57:35,048 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 089f266150a49d62d6fb34b3c6efa48f: 2023-08-07 14:57:35,048 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:57:35,048 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 14:57:35,048 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 14:57:35,048 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,049 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-08-07 14:57:35,049 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure (hbase:namespace) in zk 2023-08-07 14:57:35,051 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,051 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:35,051 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,051 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 14:57:35,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:35,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:35,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-07 14:57:35,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:35,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-08-07 14:57:35,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a218757[Count = 0] remaining members to acquire global barrier 2023-08-07 14:57:35,053 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-08-07 14:57:35,053 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,056 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,056 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-08-07 14:57:35,056 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-08-07 14:57:35,056 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase4.apache.org,39873,1691420243631' in zk 2023-08-07 14:57:35,056 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,056 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 14:57:35,059 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,059 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-08-07 14:57:35,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:35,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:35,059 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:57:35,059 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-08-07 14:57:35,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:35,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:35,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-07 14:57:35,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:35,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-07 14:57:35,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase4.apache.org,39873,1691420243631': 2023-08-07 14:57:35,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-08-07 14:57:35,065 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-08-07 14:57:35,065 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 14:57:35,065 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 14:57:35,065 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-08-07 14:57:35,065 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 14:57:35,067 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,067 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:35,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:35,067 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:57:35,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:57:35,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:35,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:57:35,068 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-07 14:57:35,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:35,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-07 14:57:35,069 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:35,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-07 14:57:35,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,075 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,075 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-07 14:57:35,075 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:57:35,075 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-07 14:57:35,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:57:35,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:35,075 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:57:35,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-08-07 14:57:35,076 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:57:35,075 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:35,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 14:57:35,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:57:35,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:57:35,076 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,076 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-07 14:57:35,076 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-07 14:57:35,078 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-08-07 14:57:35,078 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-07 14:57:45,078 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 14:57:45,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 14:57:45,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 14:57:45,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,097 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:57:45,097 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:57:45,097 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-07 14:57:45,097 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-07 14:57:45,098 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,098 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,099 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,099 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:57:45,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:57:45,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:45,099 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,100 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 14:57:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 14:57:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,100 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,100 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-07 14:57:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:57:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-07 14:57:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 14:57:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 14:57:45,101 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:45,101 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. started... 2023-08-07 14:57:45,102 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f6063f05f0012476bb995204514580c2 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-07 14:57:45,117 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/4a64bb00095d4a68b32a5b99dedbe820 2023-08-07 14:57:45,127 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/4a64bb00095d4a68b32a5b99dedbe820 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/4a64bb00095d4a68b32a5b99dedbe820 2023-08-07 14:57:45,135 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/4a64bb00095d4a68b32a5b99dedbe820, entries=1, sequenceid=5, filesize=5.8 K 2023-08-07 14:57:45,137 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for f6063f05f0012476bb995204514580c2 in 35ms, sequenceid=5, compaction requested=false 2023-08-07 14:57:45,137 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:57:45,137 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:45,137 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 14:57:45,138 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 14:57:45,138 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,138 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-07 14:57:45,138 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-07 14:57:45,139 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,139 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:45,140 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,140 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 14:57:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:45,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-07 14:57:45,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e1e14cb[Count = 0] remaining members to acquire global barrier 2023-08-07 14:57:45,141 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-07 14:57:45,142 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,143 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,143 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-07 14:57:45,143 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-07 14:57:45,143 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,39873,1691420243631' in zk 2023-08-07 14:57:45,143 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,143 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 14:57:45,145 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-07 14:57:45,145 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:57:45,145 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,145 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-07 14:57:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,39873,1691420243631': 2023-08-07 14:57:45,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-07 14:57:45,149 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-07 14:57:45,149 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 14:57:45,149 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 14:57:45,149 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,149 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 14:57:45,154 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,154 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,154 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:57:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:45,155 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:45,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:57:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:57:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,156 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,159 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,159 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,159 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:57:45,160 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,160 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:45,160 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,160 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:57:45,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:57:45,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:57:45,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 14:57:45,160 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:57:45,160 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:45,161 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:45,161 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-07 14:57:45,161 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-07 14:57:45,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:57:45,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:57:55,161 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 14:57:55,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 14:57:55,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 14:57:55,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-07 14:57:55,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,171 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:57:55,171 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:57:55,172 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-07 14:57:55,172 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-07 14:57:55,173 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,173 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,175 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,175 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:57:55,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:57:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:55,175 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,175 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 14:57:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 14:57:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,176 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-07 14:57:55,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,176 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-07 14:57:55,176 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:57:55,177 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-07 14:57:55,177 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 14:57:55,177 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 14:57:55,177 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:55,177 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. started... 2023-08-07 14:57:55,177 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f6063f05f0012476bb995204514580c2 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-07 14:57:55,194 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/27da5e07cf5448c3a740e106de35fa53 2023-08-07 14:57:55,202 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/27da5e07cf5448c3a740e106de35fa53 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/27da5e07cf5448c3a740e106de35fa53 2023-08-07 14:57:55,209 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/27da5e07cf5448c3a740e106de35fa53, entries=1, sequenceid=9, filesize=5.8 K 2023-08-07 14:57:55,210 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for f6063f05f0012476bb995204514580c2 in 33ms, sequenceid=9, compaction requested=false 2023-08-07 14:57:55,210 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:57:55,210 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:57:55,210 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 14:57:55,211 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 14:57:55,211 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,211 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-07 14:57:55,211 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-07 14:57:55,213 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,213 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:55,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:55,213 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,213 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 14:57:55,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:55,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:55,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:55,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-07 14:57:55,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b1c8d6c[Count = 0] remaining members to acquire global barrier 2023-08-07 14:57:55,215 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-07 14:57:55,215 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,217 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,217 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-07 14:57:55,217 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-07 14:57:55,217 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,217 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 14:57:55,217 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,39873,1691420243631' in zk 2023-08-07 14:57:55,219 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,219 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-07 14:57:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:55,219 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:57:55,219 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-07 14:57:55,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:55,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:55,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,39873,1691420243631': 2023-08-07 14:57:55,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-07 14:57:55,222 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-07 14:57:55,222 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 14:57:55,222 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 14:57:55,222 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,222 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 14:57:55,224 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,224 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:57:55,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:57:55,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,224 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:57:55,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:57:55,224 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:57:55,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:57:55,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:57:55,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,226 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:57:55,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,229 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,229 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,229 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:57:55,229 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:57:55,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:57:55,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 14:57:55,229 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:57:55,229 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:57:55,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:57:55,229 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:57:55,230 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,230 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,230 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-07 14:57:55,230 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:57:55,230 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-07 14:57:55,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:57:55,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:58:05,230 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 14:58:05,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 14:58:05,253 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK] 2023-08-07 14:58:05,254 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK] 2023-08-07 14:58:05,257 INFO [Listener at localhost/41565] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420244150 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420285235 2023-08-07 14:58:05,257 DEBUG [Listener at localhost/41565] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK], DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK]] 2023-08-07 14:58:05,257 DEBUG [Listener at localhost/41565] wal.AbstractFSWAL(716): hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420244150 is not closed yet, will try archiving it next time 2023-08-07 14:58:05,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 14:58:05,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-07 14:58:05,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,265 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:58:05,266 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:58:05,266 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-07 14:58:05,266 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-07 14:58:05,267 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,267 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,268 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,268 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:58:05,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:58:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:58:05,269 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,269 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 14:58:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 14:58:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-07 14:58:05,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-07 14:58:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:58:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-07 14:58:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 14:58:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 14:58:05,271 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:05,272 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. started... 2023-08-07 14:58:05,272 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f6063f05f0012476bb995204514580c2 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-07 14:58:05,284 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/48ca508231be4882a77cd20dc21803cb 2023-08-07 14:58:05,292 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/48ca508231be4882a77cd20dc21803cb as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/48ca508231be4882a77cd20dc21803cb 2023-08-07 14:58:05,300 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/48ca508231be4882a77cd20dc21803cb, entries=1, sequenceid=13, filesize=5.8 K 2023-08-07 14:58:05,301 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for f6063f05f0012476bb995204514580c2 in 29ms, sequenceid=13, compaction requested=true 2023-08-07 14:58:05,302 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:58:05,302 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:05,302 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 14:58:05,302 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 14:58:05,302 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,302 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-07 14:58:05,302 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-07 14:58:05,304 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,304 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:58:05,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:58:05,304 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,305 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 14:58:05,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:58:05,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:58:05,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:58:05,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-07 14:58:05,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c002e13[Count = 0] remaining members to acquire global barrier 2023-08-07 14:58:05,306 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-07 14:58:05,306 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,308 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,308 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-07 14:58:05,308 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,308 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 14:58:05,308 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-07 14:58:05,308 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,39873,1691420243631' in zk 2023-08-07 14:58:05,311 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-07 14:58:05,311 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,311 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:58:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:58:05,311 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-07 14:58:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:58:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:58:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:58:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:58:05,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,39873,1691420243631': 2023-08-07 14:58:05,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-07 14:58:05,314 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-07 14:58:05,314 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 14:58:05,315 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 14:58:05,315 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,315 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 14:58:05,316 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,316 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,316 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:58:05,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:58:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:58:05,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:58:05,317 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:58:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:58:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:58:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,318 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:58:05,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,321 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,321 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:58:05,321 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:58:05,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:58:05,322 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:58:05,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:58:05,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 14:58:05,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:58:05,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-07 14:58:05,322 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:58:05,322 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,322 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-07 14:58:05,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:58:05,323 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-07 14:58:05,323 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:05,323 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,323 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:05,323 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,323 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 14:58:15,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 14:58:15,325 DEBUG [Listener at localhost/41565] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 14:58:15,330 DEBUG [Listener at localhost/41565] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 14:58:15,332 DEBUG [Listener at localhost/41565] regionserver.HStore(1912): f6063f05f0012476bb995204514580c2/info is initiating minor compaction (all files) 2023-08-07 14:58:15,332 INFO [Listener at localhost/41565] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 14:58:15,332 INFO [Listener at localhost/41565] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:15,333 INFO [Listener at localhost/41565] regionserver.HRegion(2259): Starting compaction of f6063f05f0012476bb995204514580c2/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:15,333 INFO [Listener at localhost/41565] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/4a64bb00095d4a68b32a5b99dedbe820, hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/27da5e07cf5448c3a740e106de35fa53, hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/48ca508231be4882a77cd20dc21803cb] into tmpdir=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp, totalSize=17.3 K 2023-08-07 14:58:15,334 DEBUG [Listener at localhost/41565] compactions.Compactor(207): Compacting 4a64bb00095d4a68b32a5b99dedbe820, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1691420265089 2023-08-07 14:58:15,334 DEBUG [Listener at localhost/41565] compactions.Compactor(207): Compacting 27da5e07cf5448c3a740e106de35fa53, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1691420275164 2023-08-07 14:58:15,335 DEBUG [Listener at localhost/41565] compactions.Compactor(207): Compacting 48ca508231be4882a77cd20dc21803cb, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1691420285233 2023-08-07 14:58:15,355 INFO [Listener at localhost/41565] throttle.PressureAwareThroughputController(145): f6063f05f0012476bb995204514580c2#info#compaction#9 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:58:15,374 DEBUG [Listener at localhost/41565] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/22ab76ab47a4443d9522230e3caeb0b9 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/22ab76ab47a4443d9522230e3caeb0b9 2023-08-07 14:58:15,388 INFO [Listener at localhost/41565] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f6063f05f0012476bb995204514580c2/info of f6063f05f0012476bb995204514580c2 into 22ab76ab47a4443d9522230e3caeb0b9(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:58:15,388 DEBUG [Listener at localhost/41565] regionserver.HRegion(2289): Compaction status journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:58:15,404 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK] 2023-08-07 14:58:15,405 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK] 2023-08-07 14:58:15,407 INFO [Listener at localhost/41565] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420285235 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420295390 2023-08-07 14:58:15,407 DEBUG [Listener at localhost/41565] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK], DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK]] 2023-08-07 14:58:15,407 DEBUG [Listener at localhost/41565] wal.AbstractFSWAL(716): hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420285235 is not closed yet, will try archiving it next time 2023-08-07 14:58:15,407 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420244150 to hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/oldWALs/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420244150 2023-08-07 14:58:15,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 14:58:15,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-07 14:58:15,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,416 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:58:15,416 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:58:15,416 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-07 14:58:15,416 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-07 14:58:15,417 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,417 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,420 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,420 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:58:15,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:58:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:58:15,421 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,421 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 14:58:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 14:58:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,422 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-07 14:58:15,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,422 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-07 14:58:15,422 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 14:58:15,422 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-07 14:58:15,422 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 14:58:15,423 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 14:58:15,423 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:15,423 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. started... 2023-08-07 14:58:15,423 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f6063f05f0012476bb995204514580c2 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-07 14:58:15,437 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/ad353bc18dfd4750baa08856eab4b724 2023-08-07 14:58:15,446 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/ad353bc18dfd4750baa08856eab4b724 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/ad353bc18dfd4750baa08856eab4b724 2023-08-07 14:58:15,453 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/ad353bc18dfd4750baa08856eab4b724, entries=1, sequenceid=18, filesize=5.8 K 2023-08-07 14:58:15,455 INFO [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for f6063f05f0012476bb995204514580c2 in 32ms, sequenceid=18, compaction requested=false 2023-08-07 14:58:15,455 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:58:15,455 DEBUG [rs(jenkins-hbase4.apache.org,39873,1691420243631)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:15,455 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 14:58:15,456 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 14:58:15,456 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,456 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-07 14:58:15,456 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-07 14:58:15,458 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,458 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:58:15,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:58:15,458 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,458 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 14:58:15,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:58:15,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:58:15,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:58:15,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39873,1691420243631' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-07 14:58:15,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4aad15fb[Count = 0] remaining members to acquire global barrier 2023-08-07 14:58:15,460 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-07 14:58:15,460 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,462 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,462 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-07 14:58:15,462 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-07 14:58:15,462 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,39873,1691420243631' in zk 2023-08-07 14:58:15,462 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,462 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 14:58:15,464 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,464 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-07 14:58:15,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:58:15,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:58:15,464 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:58:15,464 DEBUG [member: 'jenkins-hbase4.apache.org,39873,1691420243631' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-07 14:58:15,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:58:15,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:58:15,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:58:15,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,39873,1691420243631': 2023-08-07 14:58:15,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39873,1691420243631' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-07 14:58:15,467 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-07 14:58:15,467 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 14:58:15,467 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 14:58:15,467 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,467 INFO [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 14:58:15,469 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,469 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 14:58:15,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 14:58:15,469 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:58:15,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,469 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:58:15,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 14:58:15,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:58:15,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 14:58:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,471 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 14:58:15,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,473 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,473 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 14:58:15,473 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,473 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 14:58:15,473 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,473 DEBUG [(jenkins-hbase4.apache.org,36933,1691420243583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 14:58:15,474 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-07 14:58:15,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 14:58:15,474 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:15,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-07 14:58:15,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 14:58:15,474 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:58:15,474 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,474 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-07 14:58:15,475 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-07 14:58:15,475 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-07 14:58:15,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 14:58:15,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:58:25,475 DEBUG [Listener at localhost/41565] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 14:58:25,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36933] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 14:58:25,502 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK] 2023-08-07 14:58:25,502 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK] 2023-08-07 14:58:25,504 INFO [Listener at localhost/41565] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420295390 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420305480 2023-08-07 14:58:25,504 DEBUG [Listener at localhost/41565] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33501,DS-60d74a13-8a3e-4572-b363-abaab0da7cef,DISK], DatanodeInfoWithStorage[127.0.0.1:45833,DS-8dc195d8-0fe9-4115-94b2-e64509866246,DISK]] 2023-08-07 14:58:25,504 DEBUG [Listener at localhost/41565] wal.AbstractFSWAL(716): hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420295390 is not closed yet, will try archiving it next time 2023-08-07 14:58:25,504 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-07 14:58:25,504 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/WALs/jenkins-hbase4.apache.org,39873,1691420243631/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420285235 to hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/oldWALs/jenkins-hbase4.apache.org%2C39873%2C1691420243631.1691420285235 2023-08-07 14:58:25,504 INFO [Listener at localhost/41565] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-07 14:58:25,505 DEBUG [Listener at localhost/41565] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0cf46e7e to 127.0.0.1:55126 2023-08-07 14:58:25,505 DEBUG [Listener at localhost/41565] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:58:25,506 DEBUG [Listener at localhost/41565] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 14:58:25,506 DEBUG [Listener at localhost/41565] util.JVMClusterUtil(257): Found active master hash=1924075828, stopped=false 2023-08-07 14:58:25,506 INFO [Listener at localhost/41565] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:58:25,507 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 14:58:25,507 INFO [Listener at localhost/41565] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 14:58:25,507 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 14:58:25,507 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:25,508 DEBUG [Listener at localhost/41565] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7633abe7 to 127.0.0.1:55126 2023-08-07 14:58:25,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:58:25,508 DEBUG [Listener at localhost/41565] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:58:25,509 INFO [Listener at localhost/41565] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39873,1691420243631' ***** 2023-08-07 14:58:25,509 INFO [Listener at localhost/41565] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 14:58:25,509 INFO [RS:0;jenkins-hbase4:39873] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 14:58:25,509 INFO [RS:0;jenkins-hbase4:39873] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 14:58:25,509 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 14:58:25,509 INFO [RS:0;jenkins-hbase4:39873] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 14:58:25,510 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(3305): Received CLOSE for f6063f05f0012476bb995204514580c2 2023-08-07 14:58:25,510 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(3305): Received CLOSE for 089f266150a49d62d6fb34b3c6efa48f 2023-08-07 14:58:25,510 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:25,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f6063f05f0012476bb995204514580c2, disabling compactions & flushes 2023-08-07 14:58:25,510 DEBUG [RS:0;jenkins-hbase4:39873] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f99a233 to 127.0.0.1:55126 2023-08-07 14:58:25,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:25,510 DEBUG [RS:0;jenkins-hbase4:39873] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:58:25,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:25,510 INFO [RS:0;jenkins-hbase4:39873] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 14:58:25,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. after waiting 0 ms 2023-08-07 14:58:25,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:58:25,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:25,510 INFO [RS:0;jenkins-hbase4:39873] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 14:58:25,511 INFO [RS:0;jenkins-hbase4:39873] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 14:58:25,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f6063f05f0012476bb995204514580c2 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-07 14:58:25,511 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 14:58:25,511 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-07 14:58:25,511 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1478): Online Regions={f6063f05f0012476bb995204514580c2=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2., 1588230740=hbase:meta,,1.1588230740, 089f266150a49d62d6fb34b3c6efa48f=hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f.} 2023-08-07 14:58:25,511 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 14:58:25,511 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 14:58:25,511 DEBUG [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1504): Waiting on 089f266150a49d62d6fb34b3c6efa48f, 1588230740, f6063f05f0012476bb995204514580c2 2023-08-07 14:58:25,511 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 14:58:25,512 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 14:58:25,512 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 14:58:25,512 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-08-07 14:58:25,533 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.84 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/.tmp/info/3e8bf70f40f54d5e98b887dfd9d44c17 2023-08-07 14:58:25,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/0f5cd4251b9b494494e4c8b1f8d9749c 2023-08-07 14:58:25,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/.tmp/info/0f5cd4251b9b494494e4c8b1f8d9749c as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/0f5cd4251b9b494494e4c8b1f8d9749c 2023-08-07 14:58:25,550 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/0f5cd4251b9b494494e4c8b1f8d9749c, entries=1, sequenceid=22, filesize=5.8 K 2023-08-07 14:58:25,551 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for f6063f05f0012476bb995204514580c2 in 40ms, sequenceid=22, compaction requested=true 2023-08-07 14:58:25,555 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/4a64bb00095d4a68b32a5b99dedbe820, hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/27da5e07cf5448c3a740e106de35fa53, hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/48ca508231be4882a77cd20dc21803cb] to archive 2023-08-07 14:58:25,556 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 14:58:25,559 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/4a64bb00095d4a68b32a5b99dedbe820 to hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/4a64bb00095d4a68b32a5b99dedbe820 2023-08-07 14:58:25,561 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/27da5e07cf5448c3a740e106de35fa53 to hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/27da5e07cf5448c3a740e106de35fa53 2023-08-07 14:58:25,563 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/48ca508231be4882a77cd20dc21803cb to hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/info/48ca508231be4882a77cd20dc21803cb 2023-08-07 14:58:25,569 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/.tmp/table/3fdce67df41845339cbb861b95bc7ed5 2023-08-07 14:58:25,578 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/.tmp/info/3e8bf70f40f54d5e98b887dfd9d44c17 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/info/3e8bf70f40f54d5e98b887dfd9d44c17 2023-08-07 14:58:25,584 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/info/3e8bf70f40f54d5e98b887dfd9d44c17, entries=20, sequenceid=14, filesize=7.6 K 2023-08-07 14:58:25,586 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/.tmp/table/3fdce67df41845339cbb861b95bc7ed5 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/table/3fdce67df41845339cbb861b95bc7ed5 2023-08-07 14:58:25,595 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/table/3fdce67df41845339cbb861b95bc7ed5, entries=4, sequenceid=14, filesize=4.9 K 2023-08-07 14:58:25,596 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3174, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 84ms, sequenceid=14, compaction requested=false 2023-08-07 14:58:25,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/f6063f05f0012476bb995204514580c2/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-08-07 14:58:25,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:25,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f6063f05f0012476bb995204514580c2: 2023-08-07 14:58:25,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1691420244978.f6063f05f0012476bb995204514580c2. 2023-08-07 14:58:25,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 089f266150a49d62d6fb34b3c6efa48f, disabling compactions & flushes 2023-08-07 14:58:25,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:58:25,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:58:25,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. after waiting 0 ms 2023-08-07 14:58:25,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:58:25,604 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-07 14:58:25,605 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 14:58:25,606 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 14:58:25,606 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 14:58:25,606 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 14:58:25,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/data/hbase/namespace/089f266150a49d62d6fb34b3c6efa48f/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 14:58:25,607 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:58:25,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 089f266150a49d62d6fb34b3c6efa48f: 2023-08-07 14:58:25,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691420244358.089f266150a49d62d6fb34b3c6efa48f. 2023-08-07 14:58:25,712 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39873,1691420243631; all regions closed. 2023-08-07 14:58:25,717 DEBUG [RS:0;jenkins-hbase4:39873] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/oldWALs 2023-08-07 14:58:25,718 INFO [RS:0;jenkins-hbase4:39873] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39873%2C1691420243631.meta:.meta(num 1691420244274) 2023-08-07 14:58:25,723 DEBUG [RS:0;jenkins-hbase4:39873] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/oldWALs 2023-08-07 14:58:25,723 INFO [RS:0;jenkins-hbase4:39873] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39873%2C1691420243631:(num 1691420305480) 2023-08-07 14:58:25,723 DEBUG [RS:0;jenkins-hbase4:39873] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:58:25,724 INFO [RS:0;jenkins-hbase4:39873] regionserver.LeaseManager(133): Closed leases 2023-08-07 14:58:25,724 INFO [RS:0;jenkins-hbase4:39873] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 14:58:25,724 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 14:58:25,725 INFO [RS:0;jenkins-hbase4:39873] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39873 2023-08-07 14:58:25,727 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39873,1691420243631 2023-08-07 14:58:25,727 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 14:58:25,727 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 14:58:25,729 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39873,1691420243631] 2023-08-07 14:58:25,729 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39873,1691420243631; numProcessing=1 2023-08-07 14:58:25,730 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39873,1691420243631 already deleted, retry=false 2023-08-07 14:58:25,730 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39873,1691420243631 expired; onlineServers=0 2023-08-07 14:58:25,730 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36933,1691420243583' ***** 2023-08-07 14:58:25,730 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 14:58:25,731 DEBUG [M:0;jenkins-hbase4:36933] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76908bb9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 14:58:25,731 INFO [M:0;jenkins-hbase4:36933] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36933,1691420243583 2023-08-07 14:58:25,731 INFO [M:0;jenkins-hbase4:36933] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36933,1691420243583; all regions closed. 2023-08-07 14:58:25,731 DEBUG [M:0;jenkins-hbase4:36933] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 14:58:25,731 DEBUG [M:0;jenkins-hbase4:36933] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 14:58:25,731 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 14:58:25,731 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420243880] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420243880,5,FailOnTimeoutGroup] 2023-08-07 14:58:25,731 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420243887] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420243887,5,FailOnTimeoutGroup] 2023-08-07 14:58:25,731 DEBUG [M:0;jenkins-hbase4:36933] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 14:58:25,732 INFO [M:0;jenkins-hbase4:36933] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 14:58:25,732 INFO [M:0;jenkins-hbase4:36933] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 14:58:25,732 INFO [M:0;jenkins-hbase4:36933] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 14:58:25,732 DEBUG [M:0;jenkins-hbase4:36933] master.HMaster(1512): Stopping service threads 2023-08-07 14:58:25,732 INFO [M:0;jenkins-hbase4:36933] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 14:58:25,733 ERROR [M:0;jenkins-hbase4:36933] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 14:58:25,733 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-07 14:58:25,733 INFO [M:0;jenkins-hbase4:36933] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 14:58:25,733 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:25,733 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 14:58:25,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:58:25,733 DEBUG [M:0;jenkins-hbase4:36933] zookeeper.ZKUtil(398): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-07 14:58:25,733 WARN [M:0;jenkins-hbase4:36933] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 14:58:25,734 INFO [M:0;jenkins-hbase4:36933] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 14:58:25,734 INFO [M:0;jenkins-hbase4:36933] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 14:58:25,734 DEBUG [M:0;jenkins-hbase4:36933] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 14:58:25,734 INFO [M:0;jenkins-hbase4:36933] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:25,734 DEBUG [M:0;jenkins-hbase4:36933] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:25,734 DEBUG [M:0;jenkins-hbase4:36933] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 14:58:25,735 DEBUG [M:0;jenkins-hbase4:36933] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:25,735 INFO [M:0;jenkins-hbase4:36933] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.88 KB heapSize=47.33 KB 2023-08-07 14:58:25,749 INFO [M:0;jenkins-hbase4:36933] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.88 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/64501874f34342e186e24f3f3f9f6080 2023-08-07 14:58:25,755 INFO [M:0;jenkins-hbase4:36933] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 64501874f34342e186e24f3f3f9f6080 2023-08-07 14:58:25,756 DEBUG [M:0;jenkins-hbase4:36933] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/64501874f34342e186e24f3f3f9f6080 as hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/64501874f34342e186e24f3f3f9f6080 2023-08-07 14:58:25,762 INFO [M:0;jenkins-hbase4:36933] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 64501874f34342e186e24f3f3f9f6080 2023-08-07 14:58:25,762 INFO [M:0;jenkins-hbase4:36933] regionserver.HStore(1080): Added hdfs://localhost:44739/user/jenkins/test-data/c96e605a-8361-9a94-e881-1b6222c55082/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/64501874f34342e186e24f3f3f9f6080, entries=11, sequenceid=100, filesize=6.1 K 2023-08-07 14:58:25,763 INFO [M:0;jenkins-hbase4:36933] regionserver.HRegion(2948): Finished flush of dataSize ~38.88 KB/39812, heapSize ~47.31 KB/48448, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 28ms, sequenceid=100, compaction requested=false 2023-08-07 14:58:25,764 INFO [M:0;jenkins-hbase4:36933] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:25,765 DEBUG [M:0;jenkins-hbase4:36933] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:58:25,768 INFO [M:0;jenkins-hbase4:36933] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 14:58:25,768 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 14:58:25,768 INFO [M:0;jenkins-hbase4:36933] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36933 2023-08-07 14:58:25,770 DEBUG [M:0;jenkins-hbase4:36933] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,36933,1691420243583 already deleted, retry=false 2023-08-07 14:58:25,829 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:58:25,829 INFO [RS:0;jenkins-hbase4:39873] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39873,1691420243631; zookeeper connection closed. 2023-08-07 14:58:25,830 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): regionserver:39873-0x101dfef39670001, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:58:25,830 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1951cf5e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1951cf5e 2023-08-07 14:58:25,830 INFO [Listener at localhost/41565] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-07 14:58:25,930 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:58:25,930 INFO [M:0;jenkins-hbase4:36933] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36933,1691420243583; zookeeper connection closed. 2023-08-07 14:58:25,930 DEBUG [Listener at localhost/41565-EventThread] zookeeper.ZKWatcher(600): master:36933-0x101dfef39670000, quorum=127.0.0.1:55126, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 14:58:25,931 WARN [Listener at localhost/41565] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:58:25,934 INFO [Listener at localhost/41565] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:58:26,021 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 14:58:26,039 WARN [BP-339242125-172.31.14.131-1691420242978 heartbeating to localhost/127.0.0.1:44739] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:58:26,039 WARN [BP-339242125-172.31.14.131-1691420242978 heartbeating to localhost/127.0.0.1:44739] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-339242125-172.31.14.131-1691420242978 (Datanode Uuid d228b469-a6c9-42a4-8497-5a87e67e31f7) service to localhost/127.0.0.1:44739 2023-08-07 14:58:26,040 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684/dfs/data/data3/current/BP-339242125-172.31.14.131-1691420242978] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:58:26,040 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684/dfs/data/data4/current/BP-339242125-172.31.14.131-1691420242978] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:58:26,041 WARN [Listener at localhost/41565] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 14:58:26,044 INFO [Listener at localhost/41565] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:58:26,148 WARN [BP-339242125-172.31.14.131-1691420242978 heartbeating to localhost/127.0.0.1:44739] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 14:58:26,148 WARN [BP-339242125-172.31.14.131-1691420242978 heartbeating to localhost/127.0.0.1:44739] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-339242125-172.31.14.131-1691420242978 (Datanode Uuid 735f46f1-9a6a-436b-8f24-8e9a9ad24cde) service to localhost/127.0.0.1:44739 2023-08-07 14:58:26,148 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684/dfs/data/data1/current/BP-339242125-172.31.14.131-1691420242978] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:58:26,149 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/cluster_9304c49d-c9e6-46e4-c8cf-e0eae8e70684/dfs/data/data2/current/BP-339242125-172.31.14.131-1691420242978] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 14:58:26,161 INFO [Listener at localhost/41565] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 14:58:26,275 INFO [Listener at localhost/41565] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-07 14:58:26,291 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-07 14:58:26,301 INFO [Listener at localhost/41565] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=63 (was 78), OpenFileDescriptor=439 (was 469), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=185 (was 387), ProcessCount=171 (was 171), AvailableMemoryMB=1288 (was 1935) 2023-08-07 14:58:26,309 INFO [Listener at localhost/41565] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=64, OpenFileDescriptor=439, MaxFileDescriptor=60000, SystemLoadAverage=185, ProcessCount=171, AvailableMemoryMB=1289 2023-08-07 14:58:26,309 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-07 14:58:26,309 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/hadoop.log.dir so I do NOT create it in target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7 2023-08-07 14:58:26,309 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ada65793-2523-41b4-2048-16601238cf5b/hadoop.tmp.dir so I do NOT create it in target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7 2023-08-07 14:58:26,309 INFO [Listener at localhost/41565] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8, deleteOnExit=true 2023-08-07 14:58:26,310 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-07 14:58:26,310 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/test.cache.data in system properties and HBase conf 2023-08-07 14:58:26,310 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/hadoop.tmp.dir in system properties and HBase conf 2023-08-07 14:58:26,310 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/hadoop.log.dir in system properties and HBase conf 2023-08-07 14:58:26,310 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-07 14:58:26,310 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-07 14:58:26,310 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-07 14:58:26,310 DEBUG [Listener at localhost/41565] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-07 14:58:26,311 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-07 14:58:26,311 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-07 14:58:26,311 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-07 14:58:26,311 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 14:58:26,311 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-07 14:58:26,311 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-07 14:58:26,311 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 14:58:26,312 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 14:58:26,312 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-07 14:58:26,312 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/nfs.dump.dir in system properties and HBase conf 2023-08-07 14:58:26,312 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/java.io.tmpdir in system properties and HBase conf 2023-08-07 14:58:26,312 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 14:58:26,312 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-07 14:58:26,312 INFO [Listener at localhost/41565] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-07 14:58:26,341 WARN [Listener at localhost/41565] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 14:58:26,342 WARN [Listener at localhost/41565] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 14:58:26,385 WARN [Listener at localhost/41565] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:58:26,388 INFO [Listener at localhost/41565] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:58:26,392 INFO [Listener at localhost/41565] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/java.io.tmpdir/Jetty_localhost_41713_hdfs____tuwzom/webapp 2023-08-07 14:58:26,488 INFO [Listener at localhost/41565] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41713 2023-08-07 14:58:26,493 WARN [Listener at localhost/41565] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 14:58:26,493 WARN [Listener at localhost/41565] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 14:58:26,533 WARN [Listener at localhost/40829] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:58:26,549 WARN [Listener at localhost/40829] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:58:26,552 WARN [Listener at localhost/40829] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:58:26,553 INFO [Listener at localhost/40829] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:58:26,557 INFO [Listener at localhost/40829] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/java.io.tmpdir/Jetty_localhost_34377_datanode____abhd89/webapp 2023-08-07 14:58:26,667 INFO [Listener at localhost/40829] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34377 2023-08-07 14:58:26,674 WARN [Listener at localhost/37315] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:58:26,690 WARN [Listener at localhost/37315] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 14:58:26,692 WARN [Listener at localhost/37315] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 14:58:26,694 INFO [Listener at localhost/37315] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 14:58:26,700 INFO [Listener at localhost/37315] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/java.io.tmpdir/Jetty_localhost_41313_datanode____.egf869/webapp 2023-08-07 14:58:26,794 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc4fb9c518dfa5ee4: Processing first storage report for DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34 from datanode 484c1d16-bcfc-4f7f-b9f6-b92d2a666c5d 2023-08-07 14:58:26,795 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc4fb9c518dfa5ee4: from storage DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34 node DatanodeRegistration(127.0.0.1:37281, datanodeUuid=484c1d16-bcfc-4f7f-b9f6-b92d2a666c5d, infoPort=40231, infoSecurePort=0, ipcPort=37315, storageInfo=lv=-57;cid=testClusterID;nsid=345520856;c=1691420306348), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-07 14:58:26,795 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc4fb9c518dfa5ee4: Processing first storage report for DS-8124abe6-b835-463b-8180-69b0e4483105 from datanode 484c1d16-bcfc-4f7f-b9f6-b92d2a666c5d 2023-08-07 14:58:26,795 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc4fb9c518dfa5ee4: from storage DS-8124abe6-b835-463b-8180-69b0e4483105 node DatanodeRegistration(127.0.0.1:37281, datanodeUuid=484c1d16-bcfc-4f7f-b9f6-b92d2a666c5d, infoPort=40231, infoSecurePort=0, ipcPort=37315, storageInfo=lv=-57;cid=testClusterID;nsid=345520856;c=1691420306348), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:58:26,817 INFO [Listener at localhost/37315] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41313 2023-08-07 14:58:26,825 WARN [Listener at localhost/34911] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 14:58:26,935 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd40e6aad0f589c00: Processing first storage report for DS-fc492827-cb10-4c5a-acac-045431deeca7 from datanode 1efd5c2e-a943-4d9d-8a13-1ffa9dcd0b00 2023-08-07 14:58:26,935 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd40e6aad0f589c00: from storage DS-fc492827-cb10-4c5a-acac-045431deeca7 node DatanodeRegistration(127.0.0.1:40167, datanodeUuid=1efd5c2e-a943-4d9d-8a13-1ffa9dcd0b00, infoPort=43345, infoSecurePort=0, ipcPort=34911, storageInfo=lv=-57;cid=testClusterID;nsid=345520856;c=1691420306348), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:58:26,935 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd40e6aad0f589c00: Processing first storage report for DS-9b4850b2-8adb-4e6b-8782-b6d9fdd145b3 from datanode 1efd5c2e-a943-4d9d-8a13-1ffa9dcd0b00 2023-08-07 14:58:26,935 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd40e6aad0f589c00: from storage DS-9b4850b2-8adb-4e6b-8782-b6d9fdd145b3 node DatanodeRegistration(127.0.0.1:40167, datanodeUuid=1efd5c2e-a943-4d9d-8a13-1ffa9dcd0b00, infoPort=43345, infoSecurePort=0, ipcPort=34911, storageInfo=lv=-57;cid=testClusterID;nsid=345520856;c=1691420306348), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 14:58:26,936 DEBUG [Listener at localhost/34911] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7 2023-08-07 14:58:26,941 INFO [Listener at localhost/34911] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8/zookeeper_0, clientPort=64735, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-07 14:58:26,943 INFO [Listener at localhost/34911] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64735 2023-08-07 14:58:26,943 INFO [Listener at localhost/34911] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:26,944 INFO [Listener at localhost/34911] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:26,964 INFO [Listener at localhost/34911] util.FSUtils(471): Created version file at hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd with version=8 2023-08-07 14:58:26,964 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/hbase-staging 2023-08-07 14:58:26,966 INFO [Listener at localhost/34911] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 14:58:26,966 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:58:26,967 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 14:58:26,967 INFO [Listener at localhost/34911] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 14:58:26,967 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:58:26,967 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 14:58:26,967 INFO [Listener at localhost/34911] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 14:58:26,973 INFO [Listener at localhost/34911] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44641 2023-08-07 14:58:26,973 INFO [Listener at localhost/34911] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:26,974 INFO [Listener at localhost/34911] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:26,975 INFO [Listener at localhost/34911] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44641 connecting to ZooKeeper ensemble=127.0.0.1:64735 2023-08-07 14:58:26,983 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:446410x0, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 14:58:26,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44641-0x101dff030fe0000 connected 2023-08-07 14:58:27,003 DEBUG [Listener at localhost/34911] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:58:27,003 DEBUG [Listener at localhost/34911] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:58:27,004 DEBUG [Listener at localhost/34911] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 14:58:27,005 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44641 2023-08-07 14:58:27,006 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44641 2023-08-07 14:58:27,010 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44641 2023-08-07 14:58:27,011 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44641 2023-08-07 14:58:27,011 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44641 2023-08-07 14:58:27,011 INFO [Listener at localhost/34911] master.HMaster(444): hbase.rootdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd, hbase.cluster.distributed=false 2023-08-07 14:58:27,025 INFO [Listener at localhost/34911] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 14:58:27,025 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:58:27,025 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 14:58:27,025 INFO [Listener at localhost/34911] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 14:58:27,025 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 14:58:27,025 INFO [Listener at localhost/34911] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 14:58:27,025 INFO [Listener at localhost/34911] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 14:58:27,027 INFO [Listener at localhost/34911] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38591 2023-08-07 14:58:27,027 INFO [Listener at localhost/34911] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 14:58:27,028 DEBUG [Listener at localhost/34911] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 14:58:27,029 INFO [Listener at localhost/34911] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:27,030 INFO [Listener at localhost/34911] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:27,031 INFO [Listener at localhost/34911] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38591 connecting to ZooKeeper ensemble=127.0.0.1:64735 2023-08-07 14:58:27,033 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:385910x0, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 14:58:27,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38591-0x101dff030fe0001 connected 2023-08-07 14:58:27,035 DEBUG [Listener at localhost/34911] zookeeper.ZKUtil(164): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 14:58:27,035 DEBUG [Listener at localhost/34911] zookeeper.ZKUtil(164): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 14:58:27,036 DEBUG [Listener at localhost/34911] zookeeper.ZKUtil(164): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 14:58:27,039 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38591 2023-08-07 14:58:27,039 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38591 2023-08-07 14:58:27,039 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38591 2023-08-07 14:58:27,041 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38591 2023-08-07 14:58:27,042 DEBUG [Listener at localhost/34911] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38591 2023-08-07 14:58:27,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:27,045 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 14:58:27,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:27,048 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 14:58:27,048 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 14:58:27,048 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:27,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 14:58:27,050 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,44641,1691420306966 from backup master directory 2023-08-07 14:58:27,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 14:58:27,051 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:27,051 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 14:58:27,051 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 14:58:27,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:27,067 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/hbase.id with ID: 11320b90-2871-45b3-bbb0-02681d6c6f37 2023-08-07 14:58:27,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:27,081 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:27,089 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x16620a9f to 127.0.0.1:64735 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:58:27,092 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6634223d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:58:27,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 14:58:27,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 14:58:27,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:58:27,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store-tmp 2023-08-07 14:58:27,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:27,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 14:58:27,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:27,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:27,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 14:58:27,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:27,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 14:58:27,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:58:27,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/WALs/jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:27,112 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44641%2C1691420306966, suffix=, logDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/WALs/jenkins-hbase4.apache.org,44641,1691420306966, archiveDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/oldWALs, maxLogs=10 2023-08-07 14:58:27,127 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK] 2023-08-07 14:58:27,127 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK] 2023-08-07 14:58:27,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/WALs/jenkins-hbase4.apache.org,44641,1691420306966/jenkins-hbase4.apache.org%2C44641%2C1691420306966.1691420307112 2023-08-07 14:58:27,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK], DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK]] 2023-08-07 14:58:27,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:58:27,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:27,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:58:27,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:58:27,134 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:58:27,136 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 14:58:27,136 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 14:58:27,137 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:58:27,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:58:27,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 14:58:27,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:58:27,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=688437, jitterRate=-0.12460821866989136}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:58:27,144 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 14:58:27,144 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 14:58:27,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 14:58:27,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 14:58:27,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 14:58:27,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 14:58:27,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-07 14:58:27,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 14:58:27,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-07 14:58:27,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-07 14:58:27,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 14:58:27,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 14:58:27,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-07 14:58:27,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 14:58:27,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-07 14:58:27,163 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:27,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-07 14:58:27,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-07 14:58:27,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-07 14:58:27,166 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 14:58:27,166 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:27,166 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 14:58:27,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,44641,1691420306966, sessionid=0x101dff030fe0000, setting cluster-up flag (Was=false) 2023-08-07 14:58:27,170 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:27,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-07 14:58:27,175 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:27,179 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:27,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-07 14:58:27,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:27,184 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.hbase-snapshot/.tmp 2023-08-07 14:58:27,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 14:58:27,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691420337191 2023-08-07 14:58:27,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 14:58:27,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 14:58:27,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 14:58:27,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 14:58:27,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 14:58:27,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 14:58:27,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,192 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 14:58:27,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 14:58:27,193 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-07 14:58:27,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 14:58:27,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 14:58:27,194 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 14:58:27,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 14:58:27,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 14:58:27,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420307195,5,FailOnTimeoutGroup] 2023-08-07 14:58:27,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420307195,5,FailOnTimeoutGroup] 2023-08-07 14:58:27,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 14:58:27,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,206 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 14:58:27,207 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 14:58:27,207 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd 2023-08-07 14:58:27,221 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:27,222 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 14:58:27,224 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/info 2023-08-07 14:58:27,225 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 14:58:27,225 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,226 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 14:58:27,227 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:58:27,227 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 14:58:27,228 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,228 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 14:58:27,229 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/table 2023-08-07 14:58:27,230 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 14:58:27,230 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,231 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740 2023-08-07 14:58:27,232 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740 2023-08-07 14:58:27,235 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 14:58:27,236 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 14:58:27,240 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:58:27,240 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=783589, jitterRate=-0.0036151111125946045}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 14:58:27,240 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 14:58:27,240 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 14:58:27,240 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 14:58:27,240 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 14:58:27,240 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 14:58:27,240 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 14:58:27,241 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 14:58:27,241 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 14:58:27,242 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 14:58:27,242 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-07 14:58:27,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 14:58:27,244 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 14:58:27,245 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(951): ClusterId : 11320b90-2871-45b3-bbb0-02681d6c6f37 2023-08-07 14:58:27,246 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 14:58:27,246 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 14:58:27,248 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 14:58:27,248 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 14:58:27,250 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 14:58:27,251 DEBUG [RS:0;jenkins-hbase4:38591] zookeeper.ReadOnlyZKClient(139): Connect 0x6ed56d23 to 127.0.0.1:64735 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:58:27,254 DEBUG [RS:0;jenkins-hbase4:38591] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60128e89, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:58:27,254 DEBUG [RS:0;jenkins-hbase4:38591] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@642d0653, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 14:58:27,263 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38591 2023-08-07 14:58:27,263 INFO [RS:0;jenkins-hbase4:38591] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 14:58:27,263 INFO [RS:0;jenkins-hbase4:38591] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 14:58:27,263 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 14:58:27,264 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44641,1691420306966 with isa=jenkins-hbase4.apache.org/172.31.14.131:38591, startcode=1691420307024 2023-08-07 14:58:27,264 DEBUG [RS:0;jenkins-hbase4:38591] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 14:58:27,266 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58333, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 14:58:27,267 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44641] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,268 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd 2023-08-07 14:58:27,268 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40829 2023-08-07 14:58:27,268 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 14:58:27,269 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 14:58:27,269 DEBUG [RS:0;jenkins-hbase4:38591] zookeeper.ZKUtil(162): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,269 WARN [RS:0;jenkins-hbase4:38591] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 14:58:27,270 INFO [RS:0;jenkins-hbase4:38591] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:58:27,270 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,270 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38591,1691420307024] 2023-08-07 14:58:27,273 DEBUG [RS:0;jenkins-hbase4:38591] zookeeper.ZKUtil(162): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,274 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-07 14:58:27,274 INFO [RS:0;jenkins-hbase4:38591] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 14:58:27,275 INFO [RS:0;jenkins-hbase4:38591] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 14:58:27,276 INFO [RS:0;jenkins-hbase4:38591] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 14:58:27,276 INFO [RS:0;jenkins-hbase4:38591] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,276 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 14:58:27,277 INFO [RS:0;jenkins-hbase4:38591] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,278 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,278 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,278 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,278 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,278 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,278 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 14:58:27,279 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,279 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,279 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,279 DEBUG [RS:0;jenkins-hbase4:38591] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 14:58:27,282 INFO [RS:0;jenkins-hbase4:38591] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,282 INFO [RS:0;jenkins-hbase4:38591] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,283 INFO [RS:0;jenkins-hbase4:38591] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,303 INFO [RS:0;jenkins-hbase4:38591] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 14:58:27,303 INFO [RS:0;jenkins-hbase4:38591] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38591,1691420307024-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,319 INFO [RS:0;jenkins-hbase4:38591] regionserver.Replication(203): jenkins-hbase4.apache.org,38591,1691420307024 started 2023-08-07 14:58:27,319 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38591,1691420307024, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38591, sessionid=0x101dff030fe0001 2023-08-07 14:58:27,319 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 14:58:27,319 DEBUG [RS:0;jenkins-hbase4:38591] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,319 DEBUG [RS:0;jenkins-hbase4:38591] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38591,1691420307024' 2023-08-07 14:58:27,319 DEBUG [RS:0;jenkins-hbase4:38591] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 14:58:27,320 DEBUG [RS:0;jenkins-hbase4:38591] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 14:58:27,320 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 14:58:27,320 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 14:58:27,320 DEBUG [RS:0;jenkins-hbase4:38591] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,320 DEBUG [RS:0;jenkins-hbase4:38591] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38591,1691420307024' 2023-08-07 14:58:27,320 DEBUG [RS:0;jenkins-hbase4:38591] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-07 14:58:27,321 DEBUG [RS:0;jenkins-hbase4:38591] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-07 14:58:27,321 DEBUG [RS:0;jenkins-hbase4:38591] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 14:58:27,321 INFO [RS:0;jenkins-hbase4:38591] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 14:58:27,321 INFO [RS:0;jenkins-hbase4:38591] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 14:58:27,396 DEBUG [jenkins-hbase4:44641] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 14:58:27,397 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38591,1691420307024, state=OPENING 2023-08-07 14:58:27,399 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-07 14:58:27,401 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:27,401 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38591,1691420307024}] 2023-08-07 14:58:27,401 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 14:58:27,423 INFO [RS:0;jenkins-hbase4:38591] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38591%2C1691420307024, suffix=, logDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024, archiveDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/oldWALs, maxLogs=32 2023-08-07 14:58:27,439 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK] 2023-08-07 14:58:27,439 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK] 2023-08-07 14:58:27,441 INFO [RS:0;jenkins-hbase4:38591] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420307424 2023-08-07 14:58:27,442 DEBUG [RS:0;jenkins-hbase4:38591] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK], DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK]] 2023-08-07 14:58:27,555 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,555 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 14:58:27,558 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34666, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 14:58:27,562 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 14:58:27,562 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 14:58:27,564 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38591%2C1691420307024.meta, suffix=.meta, logDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024, archiveDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/oldWALs, maxLogs=32 2023-08-07 14:58:27,578 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK] 2023-08-07 14:58:27,578 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK] 2023-08-07 14:58:27,580 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.meta.1691420307564.meta 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK], DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK]] 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 14:58:27,581 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 14:58:27,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 14:58:27,583 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 14:58:27,584 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/info 2023-08-07 14:58:27,584 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/info 2023-08-07 14:58:27,584 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 14:58:27,585 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,585 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 14:58:27,586 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:58:27,586 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/rep_barrier 2023-08-07 14:58:27,586 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 14:58:27,587 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,587 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 14:58:27,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/table 2023-08-07 14:58:27,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/table 2023-08-07 14:58:27,588 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 14:58:27,588 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,589 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740 2023-08-07 14:58:27,591 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740 2023-08-07 14:58:27,593 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 14:58:27,594 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 14:58:27,595 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=846440, jitterRate=0.07630440592765808}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 14:58:27,595 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 14:58:27,595 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691420307555 2023-08-07 14:58:27,599 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 14:58:27,599 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 14:58:27,600 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38591,1691420307024, state=OPEN 2023-08-07 14:58:27,601 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-07 14:58:27,601 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 14:58:27,603 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-07 14:58:27,603 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38591,1691420307024 in 200 msec 2023-08-07 14:58:27,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-07 14:58:27,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 361 msec 2023-08-07 14:58:27,607 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 421 msec 2023-08-07 14:58:27,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691420307607, completionTime=-1 2023-08-07 14:58:27,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-07 14:58:27,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 14:58:27,610 DEBUG [hconnection-0x4f24637f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 14:58:27,611 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34680, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 14:58:27,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-07 14:58:27,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691420367613 2023-08-07 14:58:27,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691420427613 2023-08-07 14:58:27,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 14:58:27,618 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44641,1691420306966-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44641,1691420306966-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44641,1691420306966-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:44641, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 14:58:27,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-07 14:58:27,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 14:58:27,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-07 14:58:27,620 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-07 14:58:27,621 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 14:58:27,622 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 14:58:27,624 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,624 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce empty. 2023-08-07 14:58:27,625 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,625 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-07 14:58:27,636 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-07 14:58:27,637 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 65637e06b7af61fc64e63646f6eb2cce, NAME => 'hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp 2023-08-07 14:58:27,645 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:27,645 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 65637e06b7af61fc64e63646f6eb2cce, disabling compactions & flushes 2023-08-07 14:58:27,645 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 14:58:27,645 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 14:58:27,645 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. after waiting 0 ms 2023-08-07 14:58:27,645 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 14:58:27,645 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 14:58:27,645 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 65637e06b7af61fc64e63646f6eb2cce: 2023-08-07 14:58:27,648 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 14:58:27,649 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420307649"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420307649"}]},"ts":"1691420307649"} 2023-08-07 14:58:27,651 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 14:58:27,652 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 14:58:27,652 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420307652"}]},"ts":"1691420307652"} 2023-08-07 14:58:27,654 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-07 14:58:27,660 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=65637e06b7af61fc64e63646f6eb2cce, ASSIGN}] 2023-08-07 14:58:27,662 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=65637e06b7af61fc64e63646f6eb2cce, ASSIGN 2023-08-07 14:58:27,663 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=65637e06b7af61fc64e63646f6eb2cce, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38591,1691420307024; forceNewPlan=false, retain=false 2023-08-07 14:58:27,814 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=65637e06b7af61fc64e63646f6eb2cce, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,814 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420307814"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420307814"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420307814"}]},"ts":"1691420307814"} 2023-08-07 14:58:27,816 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 65637e06b7af61fc64e63646f6eb2cce, server=jenkins-hbase4.apache.org,38591,1691420307024}] 2023-08-07 14:58:27,973 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 14:58:27,973 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 65637e06b7af61fc64e63646f6eb2cce, NAME => 'hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:58:27,973 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,973 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:27,973 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,973 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,975 INFO [StoreOpener-65637e06b7af61fc64e63646f6eb2cce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,976 DEBUG [StoreOpener-65637e06b7af61fc64e63646f6eb2cce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/info 2023-08-07 14:58:27,976 DEBUG [StoreOpener-65637e06b7af61fc64e63646f6eb2cce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/info 2023-08-07 14:58:27,977 INFO [StoreOpener-65637e06b7af61fc64e63646f6eb2cce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65637e06b7af61fc64e63646f6eb2cce columnFamilyName info 2023-08-07 14:58:27,977 INFO [StoreOpener-65637e06b7af61fc64e63646f6eb2cce-1] regionserver.HStore(310): Store=65637e06b7af61fc64e63646f6eb2cce/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:27,978 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,979 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 65637e06b7af61fc64e63646f6eb2cce 2023-08-07 14:58:27,983 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:58:27,984 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 65637e06b7af61fc64e63646f6eb2cce; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=798714, jitterRate=0.01561814546585083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:58:27,984 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 65637e06b7af61fc64e63646f6eb2cce: 2023-08-07 14:58:27,985 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce., pid=6, masterSystemTime=1691420307969 2023-08-07 14:58:27,987 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 14:58:27,987 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 14:58:27,988 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=65637e06b7af61fc64e63646f6eb2cce, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:27,988 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420307988"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420307988"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420307988"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420307988"}]},"ts":"1691420307988"} 2023-08-07 14:58:27,992 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-07 14:58:27,992 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 65637e06b7af61fc64e63646f6eb2cce, server=jenkins-hbase4.apache.org,38591,1691420307024 in 174 msec 2023-08-07 14:58:27,994 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-07 14:58:27,994 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=65637e06b7af61fc64e63646f6eb2cce, ASSIGN in 332 msec 2023-08-07 14:58:27,995 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 14:58:27,995 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420307995"}]},"ts":"1691420307995"} 2023-08-07 14:58:27,997 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-07 14:58:28,000 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 14:58:28,002 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 381 msec 2023-08-07 14:58:28,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-07 14:58:28,022 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:58:28,022 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:28,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-07 14:58:28,034 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:58:28,036 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 10 msec 2023-08-07 14:58:28,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-07 14:58:28,056 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 14:58:28,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 11 msec 2023-08-07 14:58:28,072 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-07 14:58:28,076 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-07 14:58:28,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.025sec 2023-08-07 14:58:28,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 14:58:28,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 14:58:28,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 14:58:28,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44641,1691420306966-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 14:58:28,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44641,1691420306966-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 14:58:28,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 14:58:28,145 DEBUG [Listener at localhost/34911] zookeeper.ReadOnlyZKClient(139): Connect 0x466f1429 to 127.0.0.1:64735 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 14:58:28,150 DEBUG [Listener at localhost/34911] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@631c5e2d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 14:58:28,151 DEBUG [hconnection-0x52e8f6cc-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 14:58:28,153 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34682, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 14:58:28,154 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 14:58:28,155 INFO [Listener at localhost/34911] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 14:58:28,159 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-07 14:58:28,159 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 14:58:28,160 INFO [Listener at localhost/34911] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-07 14:58:28,162 DEBUG [Listener at localhost/34911] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-07 14:58:28,164 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33602, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-07 14:58:28,166 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44641] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-07 14:58:28,166 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44641] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-07 14:58:28,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44641] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 14:58:28,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44641] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-08-07 14:58:28,171 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 14:58:28,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44641] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-08-07 14:58:28,172 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 14:58:28,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44641] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 14:58:28,174 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,174 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f empty. 2023-08-07 14:58:28,175 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,175 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-08-07 14:58:28,186 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-08-07 14:58:28,187 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3911d559d7b8f70b0620995f8c32061f, NAME => 'TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/.tmp 2023-08-07 14:58:28,195 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:28,196 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing 3911d559d7b8f70b0620995f8c32061f, disabling compactions & flushes 2023-08-07 14:58:28,196 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:28,196 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:28,196 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. after waiting 0 ms 2023-08-07 14:58:28,196 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:28,196 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:28,196 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:28,198 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 14:58:28,200 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1691420308200"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420308200"}]},"ts":"1691420308200"} 2023-08-07 14:58:28,202 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 14:58:28,203 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 14:58:28,203 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420308203"}]},"ts":"1691420308203"} 2023-08-07 14:58:28,205 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-08-07 14:58:28,208 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3911d559d7b8f70b0620995f8c32061f, ASSIGN}] 2023-08-07 14:58:28,210 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3911d559d7b8f70b0620995f8c32061f, ASSIGN 2023-08-07 14:58:28,211 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3911d559d7b8f70b0620995f8c32061f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38591,1691420307024; forceNewPlan=false, retain=false 2023-08-07 14:58:28,362 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=3911d559d7b8f70b0620995f8c32061f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:28,362 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1691420308362"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420308362"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420308362"}]},"ts":"1691420308362"} 2023-08-07 14:58:28,364 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 3911d559d7b8f70b0620995f8c32061f, server=jenkins-hbase4.apache.org,38591,1691420307024}] 2023-08-07 14:58:28,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:28,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3911d559d7b8f70b0620995f8c32061f, NAME => 'TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.', STARTKEY => '', ENDKEY => ''} 2023-08-07 14:58:28,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:28,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,524 INFO [StoreOpener-3911d559d7b8f70b0620995f8c32061f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,525 DEBUG [StoreOpener-3911d559d7b8f70b0620995f8c32061f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info 2023-08-07 14:58:28,525 DEBUG [StoreOpener-3911d559d7b8f70b0620995f8c32061f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info 2023-08-07 14:58:28,526 INFO [StoreOpener-3911d559d7b8f70b0620995f8c32061f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3911d559d7b8f70b0620995f8c32061f columnFamilyName info 2023-08-07 14:58:28,526 INFO [StoreOpener-3911d559d7b8f70b0620995f8c32061f-1] regionserver.HStore(310): Store=3911d559d7b8f70b0620995f8c32061f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:28,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:28,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 14:58:28,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3911d559d7b8f70b0620995f8c32061f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=822260, jitterRate=0.04555882513523102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:58:28,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:28,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f., pid=11, masterSystemTime=1691420308517 2023-08-07 14:58:28,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:28,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:28,537 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=3911d559d7b8f70b0620995f8c32061f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:28,538 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1691420308537"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420308537"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420308537"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420308537"}]},"ts":"1691420308537"} 2023-08-07 14:58:28,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-07 14:58:28,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 3911d559d7b8f70b0620995f8c32061f, server=jenkins-hbase4.apache.org,38591,1691420307024 in 176 msec 2023-08-07 14:58:28,543 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-07 14:58:28,544 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3911d559d7b8f70b0620995f8c32061f, ASSIGN in 334 msec 2023-08-07 14:58:28,545 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 14:58:28,545 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420308545"}]},"ts":"1691420308545"} 2023-08-07 14:58:28,546 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-08-07 14:58:28,548 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 14:58:28,550 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 382 msec 2023-08-07 14:58:31,125 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 14:58:33,274 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-07 14:58:33,275 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-07 14:58:33,275 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-08-07 14:58:38,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44641] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 14:58:38,174 INFO [Listener at localhost/34911] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-08-07 14:58:38,177 DEBUG [Listener at localhost/34911] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-08-07 14:58:38,177 DEBUG [Listener at localhost/34911] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:38,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:38,191 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3911d559d7b8f70b0620995f8c32061f 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:58:38,204 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/3d6f53153c8f4dd2a694a86012c8a4ab 2023-08-07 14:58:38,213 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/3d6f53153c8f4dd2a694a86012c8a4ab as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/3d6f53153c8f4dd2a694a86012c8a4ab 2023-08-07 14:58:38,219 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/3d6f53153c8f4dd2a694a86012c8a4ab, entries=8, sequenceid=12, filesize=13.0 K 2023-08-07 14:58:38,220 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 3911d559d7b8f70b0620995f8c32061f in 29ms, sequenceid=12, compaction requested=false 2023-08-07 14:58:38,221 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:38,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:38,222 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3911d559d7b8f70b0620995f8c32061f 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-07 14:58:38,237 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=37 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/099db06f160f459191d5701a9ec15a30 2023-08-07 14:58:38,243 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/099db06f160f459191d5701a9ec15a30 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30 2023-08-07 14:58:38,248 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30, entries=22, sequenceid=37, filesize=27.4 K 2023-08-07 14:58:38,249 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=2.05 KB/2102 for 3911d559d7b8f70b0620995f8c32061f in 27ms, sequenceid=37, compaction requested=false 2023-08-07 14:58:38,249 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:38,249 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=40.3 K, sizeToCheck=16.0 K 2023-08-07 14:58:38,249 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-07 14:58:38,251 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30 because midkey is the same as first or last row 2023-08-07 14:58:40,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:40,233 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3911d559d7b8f70b0620995f8c32061f 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:58:40,250 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=48 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/e20f623e33ef463f97e9fe6b34b5fafb 2023-08-07 14:58:40,258 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/e20f623e33ef463f97e9fe6b34b5fafb as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e20f623e33ef463f97e9fe6b34b5fafb 2023-08-07 14:58:40,264 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e20f623e33ef463f97e9fe6b34b5fafb, entries=8, sequenceid=48, filesize=13.0 K 2023-08-07 14:58:40,265 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for 3911d559d7b8f70b0620995f8c32061f in 31ms, sequenceid=48, compaction requested=true 2023-08-07 14:58:40,265 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:40,265 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=53.3 K, sizeToCheck=16.0 K 2023-08-07 14:58:40,265 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-07 14:58:40,265 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30 because midkey is the same as first or last row 2023-08-07 14:58:40,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:40,266 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:40,266 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 14:58:40,266 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3911d559d7b8f70b0620995f8c32061f 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-07 14:58:40,269 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 54576 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 14:58:40,269 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 3911d559d7b8f70b0620995f8c32061f/info is initiating minor compaction (all files) 2023-08-07 14:58:40,269 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3911d559d7b8f70b0620995f8c32061f/info in TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:40,269 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/3d6f53153c8f4dd2a694a86012c8a4ab, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e20f623e33ef463f97e9fe6b34b5fafb] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp, totalSize=53.3 K 2023-08-07 14:58:40,269 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 3d6f53153c8f4dd2a694a86012c8a4ab, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1691420318180 2023-08-07 14:58:40,270 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 099db06f160f459191d5701a9ec15a30, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=37, earliestPutTs=1691420318192 2023-08-07 14:58:40,271 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting e20f623e33ef463f97e9fe6b34b5fafb, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1691420318222 2023-08-07 14:58:40,284 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=23.61 KB at sequenceid=74 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/7359046f6fb84b22ade46e26c51befdb 2023-08-07 14:58:40,290 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3911d559d7b8f70b0620995f8c32061f#info#compaction#19 average throughput is 19.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:58:40,290 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/7359046f6fb84b22ade46e26c51befdb as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/7359046f6fb84b22ade46e26c51befdb 2023-08-07 14:58:40,297 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/7359046f6fb84b22ade46e26c51befdb, entries=23, sequenceid=74, filesize=28.4 K 2023-08-07 14:58:40,298 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~23.61 KB/24173, heapSize ~25.57 KB/26184, currentSize=3.08 KB/3153 for 3911d559d7b8f70b0620995f8c32061f in 31ms, sequenceid=74, compaction requested=false 2023-08-07 14:58:40,298 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:40,298 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=81.7 K, sizeToCheck=16.0 K 2023-08-07 14:58:40,298 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-07 14:58:40,298 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/7359046f6fb84b22ade46e26c51befdb because midkey is the same as first or last row 2023-08-07 14:58:40,310 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/fd7ca5b430ed4ebc90db48592ef2640e as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fd7ca5b430ed4ebc90db48592ef2640e 2023-08-07 14:58:40,316 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3911d559d7b8f70b0620995f8c32061f/info of 3911d559d7b8f70b0620995f8c32061f into fd7ca5b430ed4ebc90db48592ef2640e(size=43.9 K), total size for store is 72.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:58:40,316 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:40,316 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f., storeName=3911d559d7b8f70b0620995f8c32061f/info, priority=13, startTime=1691420320266; duration=0sec 2023-08-07 14:58:40,317 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=72.3 K, sizeToCheck=16.0 K 2023-08-07 14:58:40,317 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-07 14:58:40,318 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fd7ca5b430ed4ebc90db48592ef2640e because midkey is the same as first or last row 2023-08-07 14:58:40,318 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:42,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:42,278 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3911d559d7b8f70b0620995f8c32061f 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:58:42,305 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3911d559d7b8f70b0620995f8c32061f, server=jenkins-hbase4.apache.org,38591,1691420307024 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-07 14:58:42,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] ipc.CallRunner(144): callId: 99 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:34682 deadline: 1691420332305, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3911d559d7b8f70b0620995f8c32061f, server=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:42,692 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=86 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/fcf0d078a15e4e55a99643341ad7c3bb 2023-08-07 14:58:42,698 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/fcf0d078a15e4e55a99643341ad7c3bb as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fcf0d078a15e4e55a99643341ad7c3bb 2023-08-07 14:58:42,704 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fcf0d078a15e4e55a99643341ad7c3bb, entries=8, sequenceid=86, filesize=13.0 K 2023-08-07 14:58:42,705 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 3911d559d7b8f70b0620995f8c32061f in 427ms, sequenceid=86, compaction requested=true 2023-08-07 14:58:42,705 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:42,705 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=85.3 K, sizeToCheck=16.0 K 2023-08-07 14:58:42,705 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-07 14:58:42,705 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fd7ca5b430ed4ebc90db48592ef2640e because midkey is the same as first or last row 2023-08-07 14:58:42,706 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:42,706 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 14:58:42,707 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 87347 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 14:58:42,707 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 3911d559d7b8f70b0620995f8c32061f/info is initiating minor compaction (all files) 2023-08-07 14:58:42,707 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3911d559d7b8f70b0620995f8c32061f/info in TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:42,707 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fd7ca5b430ed4ebc90db48592ef2640e, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/7359046f6fb84b22ade46e26c51befdb, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fcf0d078a15e4e55a99643341ad7c3bb] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp, totalSize=85.3 K 2023-08-07 14:58:42,708 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting fd7ca5b430ed4ebc90db48592ef2640e, keycount=38, bloomtype=ROW, size=43.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1691420318180 2023-08-07 14:58:42,708 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 7359046f6fb84b22ade46e26c51befdb, keycount=23, bloomtype=ROW, size=28.4 K, encoding=NONE, compression=NONE, seqNum=74, earliestPutTs=1691420320234 2023-08-07 14:58:42,709 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting fcf0d078a15e4e55a99643341ad7c3bb, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=86, earliestPutTs=1691420320268 2023-08-07 14:58:42,721 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3911d559d7b8f70b0620995f8c32061f#info#compaction#21 average throughput is 23.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:58:42,734 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/93b934149e1143cd91ce9769cbbbc46f as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f 2023-08-07 14:58:42,739 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3911d559d7b8f70b0620995f8c32061f/info of 3911d559d7b8f70b0620995f8c32061f into 93b934149e1143cd91ce9769cbbbc46f(size=76.0 K), total size for store is 76.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:58:42,740 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:42,740 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f., storeName=3911d559d7b8f70b0620995f8c32061f/info, priority=13, startTime=1691420322705; duration=0sec 2023-08-07 14:58:42,740 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=76.0 K, sizeToCheck=16.0 K 2023-08-07 14:58:42,740 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-07 14:58:42,741 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:42,741 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:42,742 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44641] assignment.AssignmentManager(1140): Split request from jenkins-hbase4.apache.org,38591,1691420307024, parent={ENCODED => 3911d559d7b8f70b0620995f8c32061f, NAME => 'TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-08-07 14:58:42,751 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44641] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:42,758 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44641] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=3911d559d7b8f70b0620995f8c32061f, daughterA=246ce270e9d421a6fb4ef800e579756a, daughterB=03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:42,759 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=3911d559d7b8f70b0620995f8c32061f, daughterA=246ce270e9d421a6fb4ef800e579756a, daughterB=03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:42,759 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=3911d559d7b8f70b0620995f8c32061f, daughterA=246ce270e9d421a6fb4ef800e579756a, daughterB=03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:42,759 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=3911d559d7b8f70b0620995f8c32061f, daughterA=246ce270e9d421a6fb4ef800e579756a, daughterB=03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:42,766 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3911d559d7b8f70b0620995f8c32061f, UNASSIGN}] 2023-08-07 14:58:42,767 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3911d559d7b8f70b0620995f8c32061f, UNASSIGN 2023-08-07 14:58:42,768 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=3911d559d7b8f70b0620995f8c32061f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:42,768 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1691420322768"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420322768"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420322768"}]},"ts":"1691420322768"} 2023-08-07 14:58:42,770 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 3911d559d7b8f70b0620995f8c32061f, server=jenkins-hbase4.apache.org,38591,1691420307024}] 2023-08-07 14:58:42,928 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:42,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3911d559d7b8f70b0620995f8c32061f, disabling compactions & flushes 2023-08-07 14:58:42,928 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:42,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:42,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. after waiting 0 ms 2023-08-07 14:58:42,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:42,928 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 3911d559d7b8f70b0620995f8c32061f 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-08-07 14:58:42,939 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=111 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/e35bac1fbb1b4c0684e6330d7cc9acc4 2023-08-07 14:58:42,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.tmp/info/e35bac1fbb1b4c0684e6330d7cc9acc4 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e35bac1fbb1b4c0684e6330d7cc9acc4 2023-08-07 14:58:42,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e35bac1fbb1b4c0684e6330d7cc9acc4, entries=21, sequenceid=111, filesize=26.3 K 2023-08-07 14:58:42,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=0 B/0 for 3911d559d7b8f70b0620995f8c32061f in 24ms, sequenceid=111, compaction requested=false 2023-08-07 14:58:42,957 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/3d6f53153c8f4dd2a694a86012c8a4ab, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fd7ca5b430ed4ebc90db48592ef2640e, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e20f623e33ef463f97e9fe6b34b5fafb, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/7359046f6fb84b22ade46e26c51befdb, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fcf0d078a15e4e55a99643341ad7c3bb] to archive 2023-08-07 14:58:42,958 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 14:58:42,960 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/3d6f53153c8f4dd2a694a86012c8a4ab to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/3d6f53153c8f4dd2a694a86012c8a4ab 2023-08-07 14:58:42,962 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/099db06f160f459191d5701a9ec15a30 2023-08-07 14:58:42,963 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fd7ca5b430ed4ebc90db48592ef2640e to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fd7ca5b430ed4ebc90db48592ef2640e 2023-08-07 14:58:42,965 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e20f623e33ef463f97e9fe6b34b5fafb to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e20f623e33ef463f97e9fe6b34b5fafb 2023-08-07 14:58:42,966 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/7359046f6fb84b22ade46e26c51befdb to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/7359046f6fb84b22ade46e26c51befdb 2023-08-07 14:58:42,967 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fcf0d078a15e4e55a99643341ad7c3bb to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/fcf0d078a15e4e55a99643341ad7c3bb 2023-08-07 14:58:42,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/recovered.edits/114.seqid, newMaxSeqId=114, maxSeqId=1 2023-08-07 14:58:42,974 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. 2023-08-07 14:58:42,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3911d559d7b8f70b0620995f8c32061f: 2023-08-07 14:58:42,975 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:42,976 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=3911d559d7b8f70b0620995f8c32061f, regionState=CLOSED 2023-08-07 14:58:42,976 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1691420322976"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420322976"}]},"ts":"1691420322976"} 2023-08-07 14:58:42,980 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-07 14:58:42,980 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 3911d559d7b8f70b0620995f8c32061f, server=jenkins-hbase4.apache.org,38591,1691420307024 in 208 msec 2023-08-07 14:58:42,981 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-07 14:58:42,981 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3911d559d7b8f70b0620995f8c32061f, UNASSIGN in 214 msec 2023-08-07 14:58:42,993 INFO [PEWorker-2] assignment.SplitTableRegionProcedure(694): pid=12 splitting 2 storefiles, region=3911d559d7b8f70b0620995f8c32061f, threads=2 2023-08-07 14:58:42,994 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f for region: 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:42,994 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e35bac1fbb1b4c0684e6330d7cc9acc4 for region: 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:43,004 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e35bac1fbb1b4c0684e6330d7cc9acc4, top=true 2023-08-07 14:58:43,010 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/.splits/03e2a5df7cc4cdb96937269018ac2b38/info/TestLogRolling-testLogRolling=3911d559d7b8f70b0620995f8c32061f-e35bac1fbb1b4c0684e6330d7cc9acc4 for child: 03e2a5df7cc4cdb96937269018ac2b38, parent: 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:43,010 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/e35bac1fbb1b4c0684e6330d7cc9acc4 for region: 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:43,027 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f for region: 3911d559d7b8f70b0620995f8c32061f 2023-08-07 14:58:43,027 DEBUG [PEWorker-2] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 3911d559d7b8f70b0620995f8c32061f Daughter A: 1 storefiles, Daughter B: 2 storefiles. 2023-08-07 14:58:43,060 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/recovered.edits/114.seqid, newMaxSeqId=114, maxSeqId=-1 2023-08-07 14:58:43,062 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/recovered.edits/114.seqid, newMaxSeqId=114, maxSeqId=-1 2023-08-07 14:58:43,064 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1691420323064"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1691420323064"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1691420323064"}]},"ts":"1691420323064"} 2023-08-07 14:58:43,064 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691420323064"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420323064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420323064"}]},"ts":"1691420323064"} 2023-08-07 14:58:43,064 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691420323064"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420323064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420323064"}]},"ts":"1691420323064"} 2023-08-07 14:58:43,103 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38591] regionserver.HRegion(9158): Flush requested on 1588230740 2023-08-07 14:58:43,103 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-08-07 14:58:43,103 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-08-07 14:58:43,112 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=246ce270e9d421a6fb4ef800e579756a, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=03e2a5df7cc4cdb96937269018ac2b38, ASSIGN}] 2023-08-07 14:58:43,114 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=246ce270e9d421a6fb4ef800e579756a, ASSIGN 2023-08-07 14:58:43,115 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=03e2a5df7cc4cdb96937269018ac2b38, ASSIGN 2023-08-07 14:58:43,115 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=246ce270e9d421a6fb4ef800e579756a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38591,1691420307024; forceNewPlan=false, retain=false 2023-08-07 14:58:43,115 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=03e2a5df7cc4cdb96937269018ac2b38, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38591,1691420307024; forceNewPlan=false, retain=false 2023-08-07 14:58:43,117 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/.tmp/info/650d351a74fc4139a21ab993fdc40b2e 2023-08-07 14:58:43,134 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/.tmp/table/5b5b96776e7144f7b1c7caea9cea499c 2023-08-07 14:58:43,140 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/.tmp/info/650d351a74fc4139a21ab993fdc40b2e as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/info/650d351a74fc4139a21ab993fdc40b2e 2023-08-07 14:58:43,146 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/info/650d351a74fc4139a21ab993fdc40b2e, entries=29, sequenceid=17, filesize=8.6 K 2023-08-07 14:58:43,147 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/.tmp/table/5b5b96776e7144f7b1c7caea9cea499c as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/table/5b5b96776e7144f7b1c7caea9cea499c 2023-08-07 14:58:43,152 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/table/5b5b96776e7144f7b1c7caea9cea499c, entries=4, sequenceid=17, filesize=4.8 K 2023-08-07 14:58:43,153 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4934, heapSize ~8.08 KB/8272, currentSize=0 B/0 for 1588230740 in 50ms, sequenceid=17, compaction requested=false 2023-08-07 14:58:43,154 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-07 14:58:43,267 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=03e2a5df7cc4cdb96937269018ac2b38, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:43,267 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=246ce270e9d421a6fb4ef800e579756a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:43,267 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691420323267"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420323267"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420323267"}]},"ts":"1691420323267"} 2023-08-07 14:58:43,267 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691420323267"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420323267"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420323267"}]},"ts":"1691420323267"} 2023-08-07 14:58:43,269 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024}] 2023-08-07 14:58:43,270 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 246ce270e9d421a6fb4ef800e579756a, server=jenkins-hbase4.apache.org,38591,1691420307024}] 2023-08-07 14:58:43,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:58:43,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 03e2a5df7cc4cdb96937269018ac2b38, NAME => 'TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.', STARTKEY => 'row0064', ENDKEY => ''} 2023-08-07 14:58:43,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:43,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:43,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:43,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:43,426 INFO [StoreOpener-03e2a5df7cc4cdb96937269018ac2b38-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:43,427 DEBUG [StoreOpener-03e2a5df7cc4cdb96937269018ac2b38-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info 2023-08-07 14:58:43,427 DEBUG [StoreOpener-03e2a5df7cc4cdb96937269018ac2b38-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info 2023-08-07 14:58:43,428 INFO [StoreOpener-03e2a5df7cc4cdb96937269018ac2b38-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03e2a5df7cc4cdb96937269018ac2b38 columnFamilyName info 2023-08-07 14:58:43,440 DEBUG [StoreOpener-03e2a5df7cc4cdb96937269018ac2b38-1] regionserver.HStore(539): loaded hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f->hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f-top 2023-08-07 14:58:43,446 DEBUG [StoreOpener-03e2a5df7cc4cdb96937269018ac2b38-1] regionserver.HStore(539): loaded hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/TestLogRolling-testLogRolling=3911d559d7b8f70b0620995f8c32061f-e35bac1fbb1b4c0684e6330d7cc9acc4 2023-08-07 14:58:43,446 INFO [StoreOpener-03e2a5df7cc4cdb96937269018ac2b38-1] regionserver.HStore(310): Store=03e2a5df7cc4cdb96937269018ac2b38/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:43,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:43,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:43,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:58:43,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 03e2a5df7cc4cdb96937269018ac2b38; next sequenceid=115; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=782409, jitterRate=-0.005115777254104614}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:58:43,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:58:43,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., pid=17, masterSystemTime=1691420323421 2023-08-07 14:58:43,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 14:58:43,455 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-08-07 14:58:43,456 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:58:43,456 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 03e2a5df7cc4cdb96937269018ac2b38/info is initiating minor compaction (all files) 2023-08-07 14:58:43,457 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 03e2a5df7cc4cdb96937269018ac2b38/info in TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:58:43,457 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f->hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f-top, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/TestLogRolling-testLogRolling=3911d559d7b8f70b0620995f8c32061f-e35bac1fbb1b4c0684e6330d7cc9acc4] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp, totalSize=102.4 K 2023-08-07 14:58:43,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:58:43,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:58:43,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 14:58:43,458 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f, keycount=34, bloomtype=ROW, size=76.0 K, encoding=NONE, compression=NONE, seqNum=87, earliestPutTs=1691420318180 2023-08-07 14:58:43,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 246ce270e9d421a6fb4ef800e579756a, NAME => 'TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.', STARTKEY => '', ENDKEY => 'row0064'} 2023-08-07 14:58:43,458 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=03e2a5df7cc4cdb96937269018ac2b38, regionState=OPEN, openSeqNum=115, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:43,459 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691420323458"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420323458"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420323458"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420323458"}]},"ts":"1691420323458"} 2023-08-07 14:58:43,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 246ce270e9d421a6fb4ef800e579756a 2023-08-07 14:58:43,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 14:58:43,459 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=3911d559d7b8f70b0620995f8c32061f-e35bac1fbb1b4c0684e6330d7cc9acc4, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=111, earliestPutTs=1691420322279 2023-08-07 14:58:43,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 246ce270e9d421a6fb4ef800e579756a 2023-08-07 14:58:43,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 246ce270e9d421a6fb4ef800e579756a 2023-08-07 14:58:43,461 INFO [StoreOpener-246ce270e9d421a6fb4ef800e579756a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 246ce270e9d421a6fb4ef800e579756a 2023-08-07 14:58:43,463 DEBUG [StoreOpener-246ce270e9d421a6fb4ef800e579756a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info 2023-08-07 14:58:43,463 DEBUG [StoreOpener-246ce270e9d421a6fb4ef800e579756a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info 2023-08-07 14:58:43,463 INFO [StoreOpener-246ce270e9d421a6fb4ef800e579756a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 246ce270e9d421a6fb4ef800e579756a columnFamilyName info 2023-08-07 14:58:43,464 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-08-07 14:58:43,464 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 in 192 msec 2023-08-07 14:58:43,466 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=03e2a5df7cc4cdb96937269018ac2b38, ASSIGN in 352 msec 2023-08-07 14:58:43,470 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03e2a5df7cc4cdb96937269018ac2b38#info#compaction#25 average throughput is 27.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:58:43,476 DEBUG [StoreOpener-246ce270e9d421a6fb4ef800e579756a-1] regionserver.HStore(539): loaded hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f->hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f-bottom 2023-08-07 14:58:43,476 INFO [StoreOpener-246ce270e9d421a6fb4ef800e579756a-1] regionserver.HStore(310): Store=246ce270e9d421a6fb4ef800e579756a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 14:58:43,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a 2023-08-07 14:58:43,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a 2023-08-07 14:58:43,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 246ce270e9d421a6fb4ef800e579756a 2023-08-07 14:58:43,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 246ce270e9d421a6fb4ef800e579756a; next sequenceid=115; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=739543, jitterRate=-0.05962347984313965}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 14:58:43,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 246ce270e9d421a6fb4ef800e579756a: 2023-08-07 14:58:43,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a., pid=18, masterSystemTime=1691420323421 2023-08-07 14:58:43,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:43,487 DEBUG [RS:0;jenkins-hbase4:38591-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-07 14:58:43,487 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/7919650ab8694886a8fb038abd0df743 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7919650ab8694886a8fb038abd0df743 2023-08-07 14:58:43,487 INFO [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 14:58:43,487 DEBUG [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.HStore(1912): 246ce270e9d421a6fb4ef800e579756a/info is initiating minor compaction (all files) 2023-08-07 14:58:43,487 INFO [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 246ce270e9d421a6fb4ef800e579756a/info in TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 14:58:43,488 INFO [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f->hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f-bottom] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/.tmp, totalSize=76.0 K 2023-08-07 14:58:43,488 DEBUG [RS:0;jenkins-hbase4:38591-longCompactions-0] compactions.Compactor(207): Compacting 93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f, keycount=34, bloomtype=ROW, size=76.0 K, encoding=NONE, compression=NONE, seqNum=86, earliestPutTs=1691420318180 2023-08-07 14:58:43,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 14:58:43,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 14:58:43,490 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=246ce270e9d421a6fb4ef800e579756a, regionState=OPEN, openSeqNum=115, regionLocation=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:58:43,490 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691420323489"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420323489"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420323489"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420323489"}]},"ts":"1691420323489"} 2023-08-07 14:58:43,495 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-08-07 14:58:43,495 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 246ce270e9d421a6fb4ef800e579756a, server=jenkins-hbase4.apache.org,38591,1691420307024 in 221 msec 2023-08-07 14:58:43,497 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-08-07 14:58:43,498 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=246ce270e9d421a6fb4ef800e579756a, ASSIGN in 383 msec 2023-08-07 14:58:43,498 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 (all) file(s) in 03e2a5df7cc4cdb96937269018ac2b38/info of 03e2a5df7cc4cdb96937269018ac2b38 into 7919650ab8694886a8fb038abd0df743(size=32.7 K), total size for store is 32.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:58:43,498 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:58:43,498 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., storeName=03e2a5df7cc4cdb96937269018ac2b38/info, priority=14, startTime=1691420323455; duration=0sec 2023-08-07 14:58:43,498 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:43,499 INFO [RS:0;jenkins-hbase4:38591-longCompactions-0] throttle.PressureAwareThroughputController(145): 246ce270e9d421a6fb4ef800e579756a#info#compaction#26 average throughput is 15.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:58:43,500 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=3911d559d7b8f70b0620995f8c32061f, daughterA=246ce270e9d421a6fb4ef800e579756a, daughterB=03e2a5df7cc4cdb96937269018ac2b38 in 747 msec 2023-08-07 14:58:43,917 DEBUG [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/.tmp/info/372092140e8b4d21ab4962514031ffc1 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info/372092140e8b4d21ab4962514031ffc1 2023-08-07 14:58:43,924 INFO [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in 246ce270e9d421a6fb4ef800e579756a/info of 246ce270e9d421a6fb4ef800e579756a into 372092140e8b4d21ab4962514031ffc1(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:58:43,924 DEBUG [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 246ce270e9d421a6fb4ef800e579756a: 2023-08-07 14:58:43,924 INFO [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a., storeName=246ce270e9d421a6fb4ef800e579756a/info, priority=15, startTime=1691420323486; duration=0sec 2023-08-07 14:58:43,924 DEBUG [RS:0;jenkins-hbase4:38591-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:58:48,520 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 14:58:52,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] ipc.CallRunner(144): callId: 101 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:34682 deadline: 1691420342339, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1691420308166.3911d559d7b8f70b0620995f8c32061f. is not online on jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:59:14,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:14,522 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:59:14,541 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=126 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/2ad0b4022ee64a5da3eb9a5920c25ee5 2023-08-07 14:59:14,549 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/2ad0b4022ee64a5da3eb9a5920c25ee5 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/2ad0b4022ee64a5da3eb9a5920c25ee5 2023-08-07 14:59:14,553 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-07 14:59:14,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] ipc.CallRunner(144): callId: 134 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:34682 deadline: 1691420364553, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:59:14,554 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/2ad0b4022ee64a5da3eb9a5920c25ee5, entries=8, sequenceid=126, filesize=13.0 K 2023-08-07 14:59:14,555 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 03e2a5df7cc4cdb96937269018ac2b38 in 33ms, sequenceid=126, compaction requested=false 2023-08-07 14:59:14,555 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:24,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:24,596 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-07 14:59:24,609 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-07 14:59:24,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] ipc.CallRunner(144): callId: 144 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:34682 deadline: 1691420374609, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:59:24,611 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=151 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/83a9087ea34a448b853eddc774355850 2023-08-07 14:59:24,618 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/83a9087ea34a448b853eddc774355850 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/83a9087ea34a448b853eddc774355850 2023-08-07 14:59:24,623 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/83a9087ea34a448b853eddc774355850, entries=22, sequenceid=151, filesize=27.4 K 2023-08-07 14:59:24,624 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for 03e2a5df7cc4cdb96937269018ac2b38 in 28ms, sequenceid=151, compaction requested=true 2023-08-07 14:59:24,624 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:24,624 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:59:24,624 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 14:59:24,625 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 74804 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 14:59:24,625 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 03e2a5df7cc4cdb96937269018ac2b38/info is initiating minor compaction (all files) 2023-08-07 14:59:24,625 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 03e2a5df7cc4cdb96937269018ac2b38/info in TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:59:24,625 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7919650ab8694886a8fb038abd0df743, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/2ad0b4022ee64a5da3eb9a5920c25ee5, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/83a9087ea34a448b853eddc774355850] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp, totalSize=73.1 K 2023-08-07 14:59:24,626 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 7919650ab8694886a8fb038abd0df743, keycount=27, bloomtype=ROW, size=32.7 K, encoding=NONE, compression=NONE, seqNum=111, earliestPutTs=1691420320270 2023-08-07 14:59:24,626 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 2ad0b4022ee64a5da3eb9a5920c25ee5, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=126, earliestPutTs=1691420352513 2023-08-07 14:59:24,627 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 83a9087ea34a448b853eddc774355850, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=151, earliestPutTs=1691420354523 2023-08-07 14:59:24,638 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03e2a5df7cc4cdb96937269018ac2b38#info#compaction#29 average throughput is 57.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:59:24,653 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/85b4663689c84ee78720d12b364f61da as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/85b4663689c84ee78720d12b364f61da 2023-08-07 14:59:24,659 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 03e2a5df7cc4cdb96937269018ac2b38/info of 03e2a5df7cc4cdb96937269018ac2b38 into 85b4663689c84ee78720d12b364f61da(size=63.7 K), total size for store is 63.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:59:24,659 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:24,659 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., storeName=03e2a5df7cc4cdb96937269018ac2b38/info, priority=13, startTime=1691420364624; duration=0sec 2023-08-07 14:59:24,659 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:59:34,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:34,707 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:59:34,717 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=163 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/b5c26da9c6a74493b98a23736f4a5289 2023-08-07 14:59:34,724 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/b5c26da9c6a74493b98a23736f4a5289 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/b5c26da9c6a74493b98a23736f4a5289 2023-08-07 14:59:34,729 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/b5c26da9c6a74493b98a23736f4a5289, entries=8, sequenceid=163, filesize=13.0 K 2023-08-07 14:59:34,731 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=0 B/0 for 03e2a5df7cc4cdb96937269018ac2b38 in 23ms, sequenceid=163, compaction requested=false 2023-08-07 14:59:34,731 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:36,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:36,716 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:59:36,727 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=174 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/a03584994565433d8de846015ed43f78 2023-08-07 14:59:36,734 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/a03584994565433d8de846015ed43f78 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a03584994565433d8de846015ed43f78 2023-08-07 14:59:36,741 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a03584994565433d8de846015ed43f78, entries=8, sequenceid=174, filesize=13.0 K 2023-08-07 14:59:36,741 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 03e2a5df7cc4cdb96937269018ac2b38 in 25ms, sequenceid=174, compaction requested=true 2023-08-07 14:59:36,741 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:36,742 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:59:36,742 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 14:59:36,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:36,742 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-07 14:59:36,743 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 91806 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 14:59:36,743 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 03e2a5df7cc4cdb96937269018ac2b38/info is initiating minor compaction (all files) 2023-08-07 14:59:36,743 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 03e2a5df7cc4cdb96937269018ac2b38/info in TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:59:36,743 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/85b4663689c84ee78720d12b364f61da, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/b5c26da9c6a74493b98a23736f4a5289, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a03584994565433d8de846015ed43f78] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp, totalSize=89.7 K 2023-08-07 14:59:36,744 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 85b4663689c84ee78720d12b364f61da, keycount=57, bloomtype=ROW, size=63.7 K, encoding=NONE, compression=NONE, seqNum=151, earliestPutTs=1691420320270 2023-08-07 14:59:36,744 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting b5c26da9c6a74493b98a23736f4a5289, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=163, earliestPutTs=1691420364597 2023-08-07 14:59:36,745 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting a03584994565433d8de846015ed43f78, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=174, earliestPutTs=1691420376708 2023-08-07 14:59:36,755 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=199 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/f2cc22013f374b2ebe4bd8926c81dc99 2023-08-07 14:59:36,761 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/f2cc22013f374b2ebe4bd8926c81dc99 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/f2cc22013f374b2ebe4bd8926c81dc99 2023-08-07 14:59:36,766 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03e2a5df7cc4cdb96937269018ac2b38#info#compaction#33 average throughput is 24.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:59:36,769 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/f2cc22013f374b2ebe4bd8926c81dc99, entries=22, sequenceid=199, filesize=27.4 K 2023-08-07 14:59:36,771 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=2.05 KB/2102 for 03e2a5df7cc4cdb96937269018ac2b38 in 29ms, sequenceid=199, compaction requested=false 2023-08-07 14:59:36,772 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:36,780 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/c8826448a3994ee3be5b5b30a2d1a497 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/c8826448a3994ee3be5b5b30a2d1a497 2023-08-07 14:59:36,785 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 03e2a5df7cc4cdb96937269018ac2b38/info of 03e2a5df7cc4cdb96937269018ac2b38 into c8826448a3994ee3be5b5b30a2d1a497(size=80.3 K), total size for store is 107.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:59:36,785 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:36,785 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., storeName=03e2a5df7cc4cdb96937269018ac2b38/info, priority=13, startTime=1691420376742; duration=0sec 2023-08-07 14:59:36,785 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:59:38,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:38,752 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:59:38,770 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=211 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/d60e10c4350041e4b50db138bbfeb575 2023-08-07 14:59:38,776 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/d60e10c4350041e4b50db138bbfeb575 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/d60e10c4350041e4b50db138bbfeb575 2023-08-07 14:59:38,780 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-07 14:59:38,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] ipc.CallRunner(144): callId: 206 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:34682 deadline: 1691420388780, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:59:38,783 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/d60e10c4350041e4b50db138bbfeb575, entries=8, sequenceid=211, filesize=13.0 K 2023-08-07 14:59:38,784 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 03e2a5df7cc4cdb96937269018ac2b38 in 32ms, sequenceid=211, compaction requested=true 2023-08-07 14:59:38,784 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:38,784 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:59:38,784 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 14:59:38,785 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 123591 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 14:59:38,785 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 03e2a5df7cc4cdb96937269018ac2b38/info is initiating minor compaction (all files) 2023-08-07 14:59:38,785 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 03e2a5df7cc4cdb96937269018ac2b38/info in TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:59:38,785 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/c8826448a3994ee3be5b5b30a2d1a497, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/f2cc22013f374b2ebe4bd8926c81dc99, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/d60e10c4350041e4b50db138bbfeb575] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp, totalSize=120.7 K 2023-08-07 14:59:38,786 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting c8826448a3994ee3be5b5b30a2d1a497, keycount=73, bloomtype=ROW, size=80.3 K, encoding=NONE, compression=NONE, seqNum=174, earliestPutTs=1691420320270 2023-08-07 14:59:38,786 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting f2cc22013f374b2ebe4bd8926c81dc99, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=199, earliestPutTs=1691420376717 2023-08-07 14:59:38,786 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting d60e10c4350041e4b50db138bbfeb575, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=211, earliestPutTs=1691420376743 2023-08-07 14:59:38,798 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03e2a5df7cc4cdb96937269018ac2b38#info#compaction#35 average throughput is 51.62 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:59:38,817 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/a8af645e83cb4c09baa5a932d07f9f04 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a8af645e83cb4c09baa5a932d07f9f04 2023-08-07 14:59:38,823 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 03e2a5df7cc4cdb96937269018ac2b38/info of 03e2a5df7cc4cdb96937269018ac2b38 into a8af645e83cb4c09baa5a932d07f9f04(size=111.3 K), total size for store is 111.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:59:38,823 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:38,823 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., storeName=03e2a5df7cc4cdb96937269018ac2b38/info, priority=13, startTime=1691420378784; duration=0sec 2023-08-07 14:59:38,823 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:59:48,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:48,793 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-07 14:59:48,805 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=237 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/09baf9eb5c1d494c81423eb565f572ac 2023-08-07 14:59:48,811 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/09baf9eb5c1d494c81423eb565f572ac as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/09baf9eb5c1d494c81423eb565f572ac 2023-08-07 14:59:48,818 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/09baf9eb5c1d494c81423eb565f572ac, entries=22, sequenceid=237, filesize=27.4 K 2023-08-07 14:59:48,819 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=4.11 KB/4204 for 03e2a5df7cc4cdb96937269018ac2b38 in 26ms, sequenceid=237, compaction requested=false 2023-08-07 14:59:48,819 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:50,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 14:59:50,805 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 14:59:50,816 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=248 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/7cf133e3d8fe41b6af3170a96bd911c0 2023-08-07 14:59:50,825 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/7cf133e3d8fe41b6af3170a96bd911c0 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7cf133e3d8fe41b6af3170a96bd911c0 2023-08-07 14:59:50,830 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-07 14:59:50,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] ipc.CallRunner(144): callId: 238 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:34682 deadline: 1691420400830, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 14:59:50,830 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7cf133e3d8fe41b6af3170a96bd911c0, entries=8, sequenceid=248, filesize=13.0 K 2023-08-07 14:59:50,831 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 03e2a5df7cc4cdb96937269018ac2b38 in 26ms, sequenceid=248, compaction requested=true 2023-08-07 14:59:50,831 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:50,831 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 14:59:50,831 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 14:59:50,832 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 155291 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 14:59:50,832 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 03e2a5df7cc4cdb96937269018ac2b38/info is initiating minor compaction (all files) 2023-08-07 14:59:50,833 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 03e2a5df7cc4cdb96937269018ac2b38/info in TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 14:59:50,833 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a8af645e83cb4c09baa5a932d07f9f04, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/09baf9eb5c1d494c81423eb565f572ac, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7cf133e3d8fe41b6af3170a96bd911c0] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp, totalSize=151.7 K 2023-08-07 14:59:50,833 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting a8af645e83cb4c09baa5a932d07f9f04, keycount=103, bloomtype=ROW, size=111.3 K, encoding=NONE, compression=NONE, seqNum=211, earliestPutTs=1691420320270 2023-08-07 14:59:50,833 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 09baf9eb5c1d494c81423eb565f572ac, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=237, earliestPutTs=1691420378753 2023-08-07 14:59:50,834 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 7cf133e3d8fe41b6af3170a96bd911c0, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=248, earliestPutTs=1691420388795 2023-08-07 14:59:50,846 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03e2a5df7cc4cdb96937269018ac2b38#info#compaction#38 average throughput is 44.44 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 14:59:50,858 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/47c48e39b15642638e468896ecac52b2 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/47c48e39b15642638e468896ecac52b2 2023-08-07 14:59:50,863 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 03e2a5df7cc4cdb96937269018ac2b38/info of 03e2a5df7cc4cdb96937269018ac2b38 into 47c48e39b15642638e468896ecac52b2(size=142.4 K), total size for store is 142.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 14:59:50,863 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 14:59:50,863 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., storeName=03e2a5df7cc4cdb96937269018ac2b38/info, priority=13, startTime=1691420390831; duration=0sec 2023-08-07 14:59:50,863 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 15:00:00,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 15:00:00,934 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-07 15:00:00,951 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=274 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/9f46145cae4c4461b973b03a933b5162 2023-08-07 15:00:00,958 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/9f46145cae4c4461b973b03a933b5162 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9f46145cae4c4461b973b03a933b5162 2023-08-07 15:00:00,965 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9f46145cae4c4461b973b03a933b5162, entries=22, sequenceid=274, filesize=27.4 K 2023-08-07 15:00:00,966 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=6.16 KB/6306 for 03e2a5df7cc4cdb96937269018ac2b38 in 32ms, sequenceid=274, compaction requested=false 2023-08-07 15:00:00,966 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:02,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 15:00:02,948 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-07 15:00:02,959 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=285 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/9de455f0f6f54c00aa31ee9bb409d82e 2023-08-07 15:00:02,966 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/9de455f0f6f54c00aa31ee9bb409d82e as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9de455f0f6f54c00aa31ee9bb409d82e 2023-08-07 15:00:02,972 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9de455f0f6f54c00aa31ee9bb409d82e, entries=8, sequenceid=285, filesize=13.0 K 2023-08-07 15:00:02,972 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=16.42 KB/16816 for 03e2a5df7cc4cdb96937269018ac2b38 in 24ms, sequenceid=285, compaction requested=true 2023-08-07 15:00:02,972 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:02,973 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 15:00:02,973 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 15:00:02,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 15:00:02,974 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=17.45 KB heapSize=18.98 KB 2023-08-07 15:00:02,974 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 187211 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 15:00:02,974 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 03e2a5df7cc4cdb96937269018ac2b38/info is initiating minor compaction (all files) 2023-08-07 15:00:02,974 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 03e2a5df7cc4cdb96937269018ac2b38/info in TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 15:00:02,974 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/47c48e39b15642638e468896ecac52b2, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9f46145cae4c4461b973b03a933b5162, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9de455f0f6f54c00aa31ee9bb409d82e] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp, totalSize=182.8 K 2023-08-07 15:00:02,976 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 47c48e39b15642638e468896ecac52b2, keycount=133, bloomtype=ROW, size=142.4 K, encoding=NONE, compression=NONE, seqNum=248, earliestPutTs=1691420320270 2023-08-07 15:00:02,976 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 9f46145cae4c4461b973b03a933b5162, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=274, earliestPutTs=1691420390806 2023-08-07 15:00:02,977 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 9de455f0f6f54c00aa31ee9bb409d82e, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=285, earliestPutTs=1691420400935 2023-08-07 15:00:03,005 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=17.45 KB at sequenceid=305 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/605fde6c435f4908b9dc4cd92d933edc 2023-08-07 15:00:03,012 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-07 15:00:03,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] ipc.CallRunner(144): callId: 278 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:34682 deadline: 1691420413012, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=03e2a5df7cc4cdb96937269018ac2b38, server=jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 15:00:03,015 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/605fde6c435f4908b9dc4cd92d933edc as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/605fde6c435f4908b9dc4cd92d933edc 2023-08-07 15:00:03,015 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03e2a5df7cc4cdb96937269018ac2b38#info#compaction#42 average throughput is 40.84 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 15:00:03,023 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/605fde6c435f4908b9dc4cd92d933edc, entries=17, sequenceid=305, filesize=22.3 K 2023-08-07 15:00:03,025 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~17.45 KB/17867, heapSize ~18.96 KB/19416, currentSize=12.32 KB/12612 for 03e2a5df7cc4cdb96937269018ac2b38 in 51ms, sequenceid=305, compaction requested=false 2023-08-07 15:00:03,025 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:03,034 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/70ce728d85fe4927bcb8215cc2367442 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/70ce728d85fe4927bcb8215cc2367442 2023-08-07 15:00:03,040 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 03e2a5df7cc4cdb96937269018ac2b38/info of 03e2a5df7cc4cdb96937269018ac2b38 into 70ce728d85fe4927bcb8215cc2367442(size=173.4 K), total size for store is 195.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 15:00:03,040 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:03,040 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., storeName=03e2a5df7cc4cdb96937269018ac2b38/info, priority=13, startTime=1691420402973; duration=0sec 2023-08-07 15:00:03,040 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 15:00:13,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38591] regionserver.HRegion(9158): Flush requested on 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 15:00:13,055 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 03e2a5df7cc4cdb96937269018ac2b38 1/1 column families, dataSize=13.34 KB heapSize=14.57 KB 2023-08-07 15:00:13,067 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=13.34 KB at sequenceid=322 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/89bbf2a5ec54411ca2c487625cb3268c 2023-08-07 15:00:13,073 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/89bbf2a5ec54411ca2c487625cb3268c as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/89bbf2a5ec54411ca2c487625cb3268c 2023-08-07 15:00:13,078 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/89bbf2a5ec54411ca2c487625cb3268c, entries=13, sequenceid=322, filesize=18.1 K 2023-08-07 15:00:13,079 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~13.34 KB/13663, heapSize ~14.55 KB/14904, currentSize=0 B/0 for 03e2a5df7cc4cdb96937269018ac2b38 in 24ms, sequenceid=322, compaction requested=true 2023-08-07 15:00:13,079 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:13,079 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 15:00:13,079 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-07 15:00:13,080 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 218957 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-07 15:00:13,080 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1912): 03e2a5df7cc4cdb96937269018ac2b38/info is initiating minor compaction (all files) 2023-08-07 15:00:13,080 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 03e2a5df7cc4cdb96937269018ac2b38/info in TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 15:00:13,081 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/70ce728d85fe4927bcb8215cc2367442, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/605fde6c435f4908b9dc4cd92d933edc, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/89bbf2a5ec54411ca2c487625cb3268c] into tmpdir=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp, totalSize=213.8 K 2023-08-07 15:00:13,081 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 70ce728d85fe4927bcb8215cc2367442, keycount=163, bloomtype=ROW, size=173.4 K, encoding=NONE, compression=NONE, seqNum=285, earliestPutTs=1691420320270 2023-08-07 15:00:13,081 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 605fde6c435f4908b9dc4cd92d933edc, keycount=17, bloomtype=ROW, size=22.3 K, encoding=NONE, compression=NONE, seqNum=305, earliestPutTs=1691420402948 2023-08-07 15:00:13,082 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] compactions.Compactor(207): Compacting 89bbf2a5ec54411ca2c487625cb3268c, keycount=13, bloomtype=ROW, size=18.1 K, encoding=NONE, compression=NONE, seqNum=322, earliestPutTs=1691420402974 2023-08-07 15:00:13,093 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03e2a5df7cc4cdb96937269018ac2b38#info#compaction#44 average throughput is 64.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 15:00:13,110 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/.tmp/info/29e65e0677194afe88b29e0cca06a8d7 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/29e65e0677194afe88b29e0cca06a8d7 2023-08-07 15:00:13,115 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 03e2a5df7cc4cdb96937269018ac2b38/info of 03e2a5df7cc4cdb96937269018ac2b38 into 29e65e0677194afe88b29e0cca06a8d7(size=204.5 K), total size for store is 204.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 15:00:13,115 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:13,115 INFO [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., storeName=03e2a5df7cc4cdb96937269018ac2b38/info, priority=13, startTime=1691420413079; duration=0sec 2023-08-07 15:00:13,116 DEBUG [RS:0;jenkins-hbase4:38591-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 15:00:15,055 INFO [Listener at localhost/34911] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-08-07 15:00:15,080 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK] 2023-08-07 15:00:15,082 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK] 2023-08-07 15:00:15,086 INFO [Listener at localhost/34911] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420307424 with entries=309, filesize=300.61 KB; new WAL /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420415056 2023-08-07 15:00:15,086 DEBUG [Listener at localhost/34911] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK], DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK]] 2023-08-07 15:00:15,086 DEBUG [Listener at localhost/34911] wal.AbstractFSWAL(716): hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420307424 is not closed yet, will try archiving it next time 2023-08-07 15:00:15,092 DEBUG [Listener at localhost/34911] regionserver.HRegion(2446): Flush status journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:15,093 INFO [Listener at localhost/34911] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-08-07 15:00:15,111 INFO [Listener at localhost/34911] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/.tmp/info/506ee14745fd4405a4ac67fa17737eef 2023-08-07 15:00:15,119 DEBUG [Listener at localhost/34911] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/.tmp/info/506ee14745fd4405a4ac67fa17737eef as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/info/506ee14745fd4405a4ac67fa17737eef 2023-08-07 15:00:15,126 INFO [Listener at localhost/34911] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/info/506ee14745fd4405a4ac67fa17737eef, entries=16, sequenceid=24, filesize=7.0 K 2023-08-07 15:00:15,127 INFO [Listener at localhost/34911] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2312, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 34ms, sequenceid=24, compaction requested=false 2023-08-07 15:00:15,127 DEBUG [Listener at localhost/34911] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-07 15:00:15,128 INFO [Listener at localhost/34911] regionserver.HRegion(2745): Flushing 65637e06b7af61fc64e63646f6eb2cce 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-07 15:00:15,156 INFO [Listener at localhost/34911] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/.tmp/info/b3c27a6cd5e34f0aa677b280d8d0c008 2023-08-07 15:00:15,164 DEBUG [Listener at localhost/34911] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/.tmp/info/b3c27a6cd5e34f0aa677b280d8d0c008 as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/info/b3c27a6cd5e34f0aa677b280d8d0c008 2023-08-07 15:00:15,173 INFO [Listener at localhost/34911] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/info/b3c27a6cd5e34f0aa677b280d8d0c008, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 15:00:15,174 INFO [Listener at localhost/34911] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 65637e06b7af61fc64e63646f6eb2cce in 46ms, sequenceid=6, compaction requested=false 2023-08-07 15:00:15,175 DEBUG [Listener at localhost/34911] regionserver.HRegion(2446): Flush status journal for 65637e06b7af61fc64e63646f6eb2cce: 2023-08-07 15:00:15,175 DEBUG [Listener at localhost/34911] regionserver.HRegion(2446): Flush status journal for 246ce270e9d421a6fb4ef800e579756a: 2023-08-07 15:00:15,193 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK] 2023-08-07 15:00:15,194 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK] 2023-08-07 15:00:15,199 INFO [Listener at localhost/34911] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420415056 with entries=2, filesize=607 B; new WAL /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420415175 2023-08-07 15:00:15,199 DEBUG [Listener at localhost/34911] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40167,DS-fc492827-cb10-4c5a-acac-045431deeca7,DISK], DatanodeInfoWithStorage[127.0.0.1:37281,DS-9b9fad3f-ecd3-4b9c-8d94-7c895ef35d34,DISK]] 2023-08-07 15:00:15,199 DEBUG [Listener at localhost/34911] wal.AbstractFSWAL(716): hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420415056 is not closed yet, will try archiving it next time 2023-08-07 15:00:15,204 INFO [Listener at localhost/34911] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-08-07 15:00:15,204 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420307424 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/oldWALs/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420307424 2023-08-07 15:00:15,205 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/WALs/jenkins-hbase4.apache.org,38591,1691420307024/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420415056 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/oldWALs/jenkins-hbase4.apache.org%2C38591%2C1691420307024.1691420415056 2023-08-07 15:00:15,304 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-07 15:00:15,304 INFO [Listener at localhost/34911] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-07 15:00:15,304 DEBUG [Listener at localhost/34911] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x466f1429 to 127.0.0.1:64735 2023-08-07 15:00:15,304 DEBUG [Listener at localhost/34911] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:15,304 DEBUG [Listener at localhost/34911] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 15:00:15,304 DEBUG [Listener at localhost/34911] util.JVMClusterUtil(257): Found active master hash=260674228, stopped=false 2023-08-07 15:00:15,304 INFO [Listener at localhost/34911] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 15:00:15,306 INFO [Listener at localhost/34911] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 15:00:15,306 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 15:00:15,306 DEBUG [Listener at localhost/34911] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16620a9f to 127.0.0.1:64735 2023-08-07 15:00:15,306 DEBUG [Listener at localhost/34911] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:15,306 INFO [Listener at localhost/34911] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38591,1691420307024' ***** 2023-08-07 15:00:15,307 INFO [Listener at localhost/34911] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 15:00:15,306 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 15:00:15,308 INFO [RS:0;jenkins-hbase4:38591] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 15:00:15,306 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:15,308 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 15:00:15,308 INFO [RS:0;jenkins-hbase4:38591] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 15:00:15,308 INFO [RS:0;jenkins-hbase4:38591] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 15:00:15,308 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(3305): Received CLOSE for 03e2a5df7cc4cdb96937269018ac2b38 2023-08-07 15:00:15,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 15:00:15,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 15:00:15,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 03e2a5df7cc4cdb96937269018ac2b38, disabling compactions & flushes 2023-08-07 15:00:15,308 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(3305): Received CLOSE for 65637e06b7af61fc64e63646f6eb2cce 2023-08-07 15:00:15,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 15:00:15,309 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(3305): Received CLOSE for 246ce270e9d421a6fb4ef800e579756a 2023-08-07 15:00:15,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 15:00:15,309 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 15:00:15,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. after waiting 0 ms 2023-08-07 15:00:15,309 DEBUG [RS:0;jenkins-hbase4:38591] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6ed56d23 to 127.0.0.1:64735 2023-08-07 15:00:15,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 15:00:15,309 DEBUG [RS:0;jenkins-hbase4:38591] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:15,309 INFO [RS:0;jenkins-hbase4:38591] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 15:00:15,309 INFO [RS:0;jenkins-hbase4:38591] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 15:00:15,309 INFO [RS:0;jenkins-hbase4:38591] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 15:00:15,309 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 15:00:15,309 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-07 15:00:15,310 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1478): Online Regions={03e2a5df7cc4cdb96937269018ac2b38=TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38., 1588230740=hbase:meta,,1.1588230740, 65637e06b7af61fc64e63646f6eb2cce=hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce., 246ce270e9d421a6fb4ef800e579756a=TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.} 2023-08-07 15:00:15,310 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 15:00:15,310 DEBUG [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1504): Waiting on 03e2a5df7cc4cdb96937269018ac2b38, 1588230740, 246ce270e9d421a6fb4ef800e579756a, 65637e06b7af61fc64e63646f6eb2cce 2023-08-07 15:00:15,310 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 15:00:15,311 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 15:00:15,311 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 15:00:15,311 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 15:00:15,332 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f->hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f-top, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7919650ab8694886a8fb038abd0df743, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/TestLogRolling-testLogRolling=3911d559d7b8f70b0620995f8c32061f-e35bac1fbb1b4c0684e6330d7cc9acc4, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/2ad0b4022ee64a5da3eb9a5920c25ee5, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/85b4663689c84ee78720d12b364f61da, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/83a9087ea34a448b853eddc774355850, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/b5c26da9c6a74493b98a23736f4a5289, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/c8826448a3994ee3be5b5b30a2d1a497, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a03584994565433d8de846015ed43f78, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/f2cc22013f374b2ebe4bd8926c81dc99, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a8af645e83cb4c09baa5a932d07f9f04, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/d60e10c4350041e4b50db138bbfeb575, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/09baf9eb5c1d494c81423eb565f572ac, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/47c48e39b15642638e468896ecac52b2, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7cf133e3d8fe41b6af3170a96bd911c0, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9f46145cae4c4461b973b03a933b5162, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/70ce728d85fe4927bcb8215cc2367442, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9de455f0f6f54c00aa31ee9bb409d82e, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/605fde6c435f4908b9dc4cd92d933edc, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/89bbf2a5ec54411ca2c487625cb3268c] to archive 2023-08-07 15:00:15,333 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 15:00:15,334 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-08-07 15:00:15,335 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 15:00:15,335 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 15:00:15,335 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 15:00:15,336 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 15:00:15,336 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f 2023-08-07 15:00:15,337 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7919650ab8694886a8fb038abd0df743 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7919650ab8694886a8fb038abd0df743 2023-08-07 15:00:15,339 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/TestLogRolling-testLogRolling=3911d559d7b8f70b0620995f8c32061f-e35bac1fbb1b4c0684e6330d7cc9acc4 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/TestLogRolling-testLogRolling=3911d559d7b8f70b0620995f8c32061f-e35bac1fbb1b4c0684e6330d7cc9acc4 2023-08-07 15:00:15,340 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/2ad0b4022ee64a5da3eb9a5920c25ee5 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/2ad0b4022ee64a5da3eb9a5920c25ee5 2023-08-07 15:00:15,342 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/85b4663689c84ee78720d12b364f61da to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/85b4663689c84ee78720d12b364f61da 2023-08-07 15:00:15,343 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/83a9087ea34a448b853eddc774355850 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/83a9087ea34a448b853eddc774355850 2023-08-07 15:00:15,345 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/b5c26da9c6a74493b98a23736f4a5289 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/b5c26da9c6a74493b98a23736f4a5289 2023-08-07 15:00:15,346 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/c8826448a3994ee3be5b5b30a2d1a497 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/c8826448a3994ee3be5b5b30a2d1a497 2023-08-07 15:00:15,347 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a03584994565433d8de846015ed43f78 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a03584994565433d8de846015ed43f78 2023-08-07 15:00:15,348 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/f2cc22013f374b2ebe4bd8926c81dc99 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/f2cc22013f374b2ebe4bd8926c81dc99 2023-08-07 15:00:15,349 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a8af645e83cb4c09baa5a932d07f9f04 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/a8af645e83cb4c09baa5a932d07f9f04 2023-08-07 15:00:15,351 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/d60e10c4350041e4b50db138bbfeb575 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/d60e10c4350041e4b50db138bbfeb575 2023-08-07 15:00:15,352 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/09baf9eb5c1d494c81423eb565f572ac to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/09baf9eb5c1d494c81423eb565f572ac 2023-08-07 15:00:15,353 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/47c48e39b15642638e468896ecac52b2 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/47c48e39b15642638e468896ecac52b2 2023-08-07 15:00:15,355 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7cf133e3d8fe41b6af3170a96bd911c0 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/7cf133e3d8fe41b6af3170a96bd911c0 2023-08-07 15:00:15,356 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9f46145cae4c4461b973b03a933b5162 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9f46145cae4c4461b973b03a933b5162 2023-08-07 15:00:15,357 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/70ce728d85fe4927bcb8215cc2367442 to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/70ce728d85fe4927bcb8215cc2367442 2023-08-07 15:00:15,359 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9de455f0f6f54c00aa31ee9bb409d82e to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/9de455f0f6f54c00aa31ee9bb409d82e 2023-08-07 15:00:15,360 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/605fde6c435f4908b9dc4cd92d933edc to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/605fde6c435f4908b9dc4cd92d933edc 2023-08-07 15:00:15,361 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/89bbf2a5ec54411ca2c487625cb3268c to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/info/89bbf2a5ec54411ca2c487625cb3268c 2023-08-07 15:00:15,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/03e2a5df7cc4cdb96937269018ac2b38/recovered.edits/327.seqid, newMaxSeqId=327, maxSeqId=114 2023-08-07 15:00:15,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 15:00:15,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 03e2a5df7cc4cdb96937269018ac2b38: 2023-08-07 15:00:15,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1691420322751.03e2a5df7cc4cdb96937269018ac2b38. 2023-08-07 15:00:15,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 65637e06b7af61fc64e63646f6eb2cce, disabling compactions & flushes 2023-08-07 15:00:15,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 15:00:15,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 15:00:15,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. after waiting 0 ms 2023-08-07 15:00:15,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 15:00:15,372 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/hbase/namespace/65637e06b7af61fc64e63646f6eb2cce/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 15:00:15,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 15:00:15,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 65637e06b7af61fc64e63646f6eb2cce: 2023-08-07 15:00:15,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691420307619.65637e06b7af61fc64e63646f6eb2cce. 2023-08-07 15:00:15,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 246ce270e9d421a6fb4ef800e579756a, disabling compactions & flushes 2023-08-07 15:00:15,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 15:00:15,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 15:00:15,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. after waiting 0 ms 2023-08-07 15:00:15,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 15:00:15,374 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f->hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/3911d559d7b8f70b0620995f8c32061f/info/93b934149e1143cd91ce9769cbbbc46f-bottom] to archive 2023-08-07 15:00:15,375 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 15:00:15,377 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f to hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/archive/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/info/93b934149e1143cd91ce9769cbbbc46f.3911d559d7b8f70b0620995f8c32061f 2023-08-07 15:00:15,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/data/default/TestLogRolling-testLogRolling/246ce270e9d421a6fb4ef800e579756a/recovered.edits/119.seqid, newMaxSeqId=119, maxSeqId=114 2023-08-07 15:00:15,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 15:00:15,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 246ce270e9d421a6fb4ef800e579756a: 2023-08-07 15:00:15,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1691420322751.246ce270e9d421a6fb4ef800e579756a. 2023-08-07 15:00:15,510 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38591,1691420307024; all regions closed. 2023-08-07 15:00:15,517 DEBUG [RS:0;jenkins-hbase4:38591] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/oldWALs 2023-08-07 15:00:15,517 INFO [RS:0;jenkins-hbase4:38591] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38591%2C1691420307024.meta:.meta(num 1691420307564) 2023-08-07 15:00:15,522 DEBUG [RS:0;jenkins-hbase4:38591] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/oldWALs 2023-08-07 15:00:15,522 INFO [RS:0;jenkins-hbase4:38591] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38591%2C1691420307024:(num 1691420415175) 2023-08-07 15:00:15,522 DEBUG [RS:0;jenkins-hbase4:38591] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:15,522 INFO [RS:0;jenkins-hbase4:38591] regionserver.LeaseManager(133): Closed leases 2023-08-07 15:00:15,523 INFO [RS:0;jenkins-hbase4:38591] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-07 15:00:15,523 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 15:00:15,523 INFO [RS:0;jenkins-hbase4:38591] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38591 2023-08-07 15:00:15,525 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38591,1691420307024 2023-08-07 15:00:15,525 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 15:00:15,525 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 15:00:15,527 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38591,1691420307024] 2023-08-07 15:00:15,527 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38591,1691420307024; numProcessing=1 2023-08-07 15:00:15,529 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38591,1691420307024 already deleted, retry=false 2023-08-07 15:00:15,529 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38591,1691420307024 expired; onlineServers=0 2023-08-07 15:00:15,529 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44641,1691420306966' ***** 2023-08-07 15:00:15,529 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 15:00:15,530 DEBUG [M:0;jenkins-hbase4:44641] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@bb6079d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 15:00:15,530 INFO [M:0;jenkins-hbase4:44641] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44641,1691420306966 2023-08-07 15:00:15,530 INFO [M:0;jenkins-hbase4:44641] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44641,1691420306966; all regions closed. 2023-08-07 15:00:15,530 DEBUG [M:0;jenkins-hbase4:44641] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:15,530 DEBUG [M:0;jenkins-hbase4:44641] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 15:00:15,530 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 15:00:15,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420307195] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420307195,5,FailOnTimeoutGroup] 2023-08-07 15:00:15,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420307195] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420307195,5,FailOnTimeoutGroup] 2023-08-07 15:00:15,530 DEBUG [M:0;jenkins-hbase4:44641] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 15:00:15,531 INFO [M:0;jenkins-hbase4:44641] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 15:00:15,531 INFO [M:0;jenkins-hbase4:44641] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 15:00:15,531 INFO [M:0;jenkins-hbase4:44641] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 15:00:15,531 DEBUG [M:0;jenkins-hbase4:44641] master.HMaster(1512): Stopping service threads 2023-08-07 15:00:15,531 INFO [M:0;jenkins-hbase4:44641] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 15:00:15,532 ERROR [M:0;jenkins-hbase4:44641] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 15:00:15,532 INFO [M:0;jenkins-hbase4:44641] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 15:00:15,532 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-07 15:00:15,532 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 15:00:15,532 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:15,532 DEBUG [M:0;jenkins-hbase4:44641] zookeeper.ZKUtil(398): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-07 15:00:15,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 15:00:15,532 WARN [M:0;jenkins-hbase4:44641] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 15:00:15,533 INFO [M:0;jenkins-hbase4:44641] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 15:00:15,533 INFO [M:0;jenkins-hbase4:44641] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 15:00:15,533 DEBUG [M:0;jenkins-hbase4:44641] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 15:00:15,533 INFO [M:0;jenkins-hbase4:44641] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:15,533 DEBUG [M:0;jenkins-hbase4:44641] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:15,533 DEBUG [M:0;jenkins-hbase4:44641] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 15:00:15,533 DEBUG [M:0;jenkins-hbase4:44641] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:15,534 INFO [M:0;jenkins-hbase4:44641] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.69 KB heapSize=78.42 KB 2023-08-07 15:00:15,546 INFO [M:0;jenkins-hbase4:44641] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.69 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/604b9565149f4a00b18fa8f199bf274c 2023-08-07 15:00:15,553 INFO [M:0;jenkins-hbase4:44641] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 604b9565149f4a00b18fa8f199bf274c 2023-08-07 15:00:15,554 DEBUG [M:0;jenkins-hbase4:44641] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/604b9565149f4a00b18fa8f199bf274c as hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/604b9565149f4a00b18fa8f199bf274c 2023-08-07 15:00:15,564 INFO [M:0;jenkins-hbase4:44641] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 604b9565149f4a00b18fa8f199bf274c 2023-08-07 15:00:15,564 INFO [M:0;jenkins-hbase4:44641] regionserver.HStore(1080): Added hdfs://localhost:40829/user/jenkins/test-data/961edb3c-e9d4-3eae-c5ee-16e6d6b351dd/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/604b9565149f4a00b18fa8f199bf274c, entries=18, sequenceid=160, filesize=6.9 K 2023-08-07 15:00:15,565 INFO [M:0;jenkins-hbase4:44641] regionserver.HRegion(2948): Finished flush of dataSize ~64.69 KB/66244, heapSize ~78.41 KB/80288, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 32ms, sequenceid=160, compaction requested=false 2023-08-07 15:00:15,567 INFO [M:0;jenkins-hbase4:44641] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:15,567 DEBUG [M:0;jenkins-hbase4:44641] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 15:00:15,571 INFO [M:0;jenkins-hbase4:44641] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 15:00:15,571 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 15:00:15,572 INFO [M:0;jenkins-hbase4:44641] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44641 2023-08-07 15:00:15,574 DEBUG [M:0;jenkins-hbase4:44641] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,44641,1691420306966 already deleted, retry=false 2023-08-07 15:00:15,627 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:15,627 INFO [RS:0;jenkins-hbase4:38591] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38591,1691420307024; zookeeper connection closed. 2023-08-07 15:00:15,627 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): regionserver:38591-0x101dff030fe0001, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:15,628 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@54f48f77] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@54f48f77 2023-08-07 15:00:15,628 INFO [Listener at localhost/34911] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-07 15:00:15,727 INFO [M:0;jenkins-hbase4:44641] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44641,1691420306966; zookeeper connection closed. 2023-08-07 15:00:15,728 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:15,728 DEBUG [Listener at localhost/34911-EventThread] zookeeper.ZKWatcher(600): master:44641-0x101dff030fe0000, quorum=127.0.0.1:64735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:15,729 WARN [Listener at localhost/34911] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 15:00:15,805 INFO [Listener at localhost/34911] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 15:00:15,912 WARN [BP-1489505544-172.31.14.131-1691420306348 heartbeating to localhost/127.0.0.1:40829] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 15:00:15,912 WARN [BP-1489505544-172.31.14.131-1691420306348 heartbeating to localhost/127.0.0.1:40829] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1489505544-172.31.14.131-1691420306348 (Datanode Uuid 1efd5c2e-a943-4d9d-8a13-1ffa9dcd0b00) service to localhost/127.0.0.1:40829 2023-08-07 15:00:15,913 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8/dfs/data/data3/current/BP-1489505544-172.31.14.131-1691420306348] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:15,913 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8/dfs/data/data4/current/BP-1489505544-172.31.14.131-1691420306348] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:15,915 WARN [Listener at localhost/34911] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 15:00:15,919 INFO [Listener at localhost/34911] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 15:00:16,024 WARN [BP-1489505544-172.31.14.131-1691420306348 heartbeating to localhost/127.0.0.1:40829] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 15:00:16,024 WARN [BP-1489505544-172.31.14.131-1691420306348 heartbeating to localhost/127.0.0.1:40829] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1489505544-172.31.14.131-1691420306348 (Datanode Uuid 484c1d16-bcfc-4f7f-b9f6-b92d2a666c5d) service to localhost/127.0.0.1:40829 2023-08-07 15:00:16,024 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8/dfs/data/data1/current/BP-1489505544-172.31.14.131-1691420306348] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:16,025 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/cluster_b9845d00-6be7-45fc-6e76-ea9afe5d31c8/dfs/data/data2/current/BP-1489505544-172.31.14.131-1691420306348] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:16,035 INFO [Listener at localhost/34911] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 15:00:16,152 INFO [Listener at localhost/34911] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-07 15:00:16,195 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-07 15:00:16,209 INFO [Listener at localhost/34911] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=76 (was 64) - Thread LEAK? -, OpenFileDescriptor=467 (was 439) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=220 (was 185) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=1498 (was 1289) - AvailableMemoryMB LEAK? - 2023-08-07 15:00:16,222 INFO [Listener at localhost/34911] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=77, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=220, ProcessCount=171, AvailableMemoryMB=1498 2023-08-07 15:00:16,223 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-07 15:00:16,223 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/hadoop.log.dir so I do NOT create it in target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4 2023-08-07 15:00:16,223 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/386a7085-11a5-ba67-8292-21350eaa45f7/hadoop.tmp.dir so I do NOT create it in target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4 2023-08-07 15:00:16,223 INFO [Listener at localhost/34911] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305, deleteOnExit=true 2023-08-07 15:00:16,223 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-07 15:00:16,223 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/test.cache.data in system properties and HBase conf 2023-08-07 15:00:16,223 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/hadoop.tmp.dir in system properties and HBase conf 2023-08-07 15:00:16,224 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/hadoop.log.dir in system properties and HBase conf 2023-08-07 15:00:16,224 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-07 15:00:16,224 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-07 15:00:16,224 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-07 15:00:16,224 DEBUG [Listener at localhost/34911] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-07 15:00:16,224 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-07 15:00:16,224 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-07 15:00:16,224 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/nfs.dump.dir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/java.io.tmpdir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 15:00:16,225 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-07 15:00:16,226 INFO [Listener at localhost/34911] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-07 15:00:16,237 WARN [Listener at localhost/34911] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 15:00:16,237 WARN [Listener at localhost/34911] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 15:00:16,289 WARN [Listener at localhost/34911] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 15:00:16,292 INFO [Listener at localhost/34911] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 15:00:16,299 INFO [Listener at localhost/34911] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/java.io.tmpdir/Jetty_localhost_38481_hdfs____xs5cnm/webapp 2023-08-07 15:00:16,398 INFO [Listener at localhost/34911] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38481 2023-08-07 15:00:16,457 WARN [Listener at localhost/34911] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 15:00:16,457 WARN [Listener at localhost/34911] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 15:00:16,544 WARN [Listener at localhost/36247] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 15:00:16,570 WARN [Listener at localhost/36247] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 15:00:16,573 WARN [Listener at localhost/36247] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 15:00:16,574 INFO [Listener at localhost/36247] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 15:00:16,580 INFO [Listener at localhost/36247] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/java.io.tmpdir/Jetty_localhost_35465_datanode____.q5nz2g/webapp 2023-08-07 15:00:16,673 INFO [Listener at localhost/36247] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35465 2023-08-07 15:00:16,680 WARN [Listener at localhost/38435] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 15:00:16,690 WARN [Listener at localhost/38435] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 15:00:16,692 WARN [Listener at localhost/38435] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 15:00:16,694 INFO [Listener at localhost/38435] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 15:00:16,703 INFO [Listener at localhost/38435] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/java.io.tmpdir/Jetty_localhost_34049_datanode____lxjk4b/webapp 2023-08-07 15:00:16,783 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xef2acbb68c572a8b: Processing first storage report for DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb from datanode 8263d490-5de5-4462-a84f-1a6d46cdf892 2023-08-07 15:00:16,783 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xef2acbb68c572a8b: from storage DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb node DatanodeRegistration(127.0.0.1:45241, datanodeUuid=8263d490-5de5-4462-a84f-1a6d46cdf892, infoPort=43281, infoSecurePort=0, ipcPort=38435, storageInfo=lv=-57;cid=testClusterID;nsid=740437528;c=1691420416246), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 15:00:16,783 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xef2acbb68c572a8b: Processing first storage report for DS-99fc94b3-400a-476b-b3d7-d1fa91233889 from datanode 8263d490-5de5-4462-a84f-1a6d46cdf892 2023-08-07 15:00:16,783 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xef2acbb68c572a8b: from storage DS-99fc94b3-400a-476b-b3d7-d1fa91233889 node DatanodeRegistration(127.0.0.1:45241, datanodeUuid=8263d490-5de5-4462-a84f-1a6d46cdf892, infoPort=43281, infoSecurePort=0, ipcPort=38435, storageInfo=lv=-57;cid=testClusterID;nsid=740437528;c=1691420416246), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 15:00:16,800 INFO [Listener at localhost/38435] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34049 2023-08-07 15:00:16,807 WARN [Listener at localhost/41271] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 15:00:16,904 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xed4d6ddb1231df9c: Processing first storage report for DS-fde23b82-624a-473b-a0ea-70024d38836d from datanode 46af531b-4b0b-4670-bca4-770962143850 2023-08-07 15:00:16,904 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xed4d6ddb1231df9c: from storage DS-fde23b82-624a-473b-a0ea-70024d38836d node DatanodeRegistration(127.0.0.1:33099, datanodeUuid=46af531b-4b0b-4670-bca4-770962143850, infoPort=35379, infoSecurePort=0, ipcPort=41271, storageInfo=lv=-57;cid=testClusterID;nsid=740437528;c=1691420416246), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 15:00:16,904 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xed4d6ddb1231df9c: Processing first storage report for DS-40176ca1-4ce5-4059-9512-50cb2467e742 from datanode 46af531b-4b0b-4670-bca4-770962143850 2023-08-07 15:00:16,904 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xed4d6ddb1231df9c: from storage DS-40176ca1-4ce5-4059-9512-50cb2467e742 node DatanodeRegistration(127.0.0.1:33099, datanodeUuid=46af531b-4b0b-4670-bca4-770962143850, infoPort=35379, infoSecurePort=0, ipcPort=41271, storageInfo=lv=-57;cid=testClusterID;nsid=740437528;c=1691420416246), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 15:00:16,914 DEBUG [Listener at localhost/41271] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4 2023-08-07 15:00:16,916 INFO [Listener at localhost/41271] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305/zookeeper_0, clientPort=60523, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-07 15:00:16,917 INFO [Listener at localhost/41271] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=60523 2023-08-07 15:00:16,918 INFO [Listener at localhost/41271] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:16,918 INFO [Listener at localhost/41271] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:16,930 INFO [Listener at localhost/41271] util.FSUtils(471): Created version file at hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547 with version=8 2023-08-07 15:00:16,930 INFO [Listener at localhost/41271] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:44501/user/jenkins/test-data/6d3b9657-8c09-b88a-be89-0218d84c2da4/hbase-staging 2023-08-07 15:00:16,932 INFO [Listener at localhost/41271] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 15:00:16,932 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 15:00:16,932 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 15:00:16,932 INFO [Listener at localhost/41271] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 15:00:16,932 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 15:00:16,932 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 15:00:16,932 INFO [Listener at localhost/41271] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 15:00:16,933 INFO [Listener at localhost/41271] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46343 2023-08-07 15:00:16,934 INFO [Listener at localhost/41271] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:16,935 INFO [Listener at localhost/41271] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:16,936 INFO [Listener at localhost/41271] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46343 connecting to ZooKeeper ensemble=127.0.0.1:60523 2023-08-07 15:00:16,942 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:463430x0, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 15:00:16,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:46343-0x101dff1de970000 connected 2023-08-07 15:00:16,959 DEBUG [Listener at localhost/41271] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 15:00:16,959 DEBUG [Listener at localhost/41271] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 15:00:16,960 DEBUG [Listener at localhost/41271] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 15:00:16,960 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46343 2023-08-07 15:00:16,960 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46343 2023-08-07 15:00:16,961 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46343 2023-08-07 15:00:16,961 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46343 2023-08-07 15:00:16,961 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46343 2023-08-07 15:00:16,961 INFO [Listener at localhost/41271] master.HMaster(444): hbase.rootdir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547, hbase.cluster.distributed=false 2023-08-07 15:00:16,976 INFO [Listener at localhost/41271] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 15:00:16,976 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 15:00:16,976 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 15:00:16,976 INFO [Listener at localhost/41271] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 15:00:16,976 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 15:00:16,976 INFO [Listener at localhost/41271] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 15:00:16,976 INFO [Listener at localhost/41271] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 15:00:16,979 INFO [Listener at localhost/41271] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34479 2023-08-07 15:00:16,979 INFO [Listener at localhost/41271] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 15:00:16,980 DEBUG [Listener at localhost/41271] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 15:00:16,981 INFO [Listener at localhost/41271] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:16,982 INFO [Listener at localhost/41271] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:16,984 INFO [Listener at localhost/41271] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34479 connecting to ZooKeeper ensemble=127.0.0.1:60523 2023-08-07 15:00:16,989 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:344790x0, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 15:00:16,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34479-0x101dff1de970001 connected 2023-08-07 15:00:16,990 DEBUG [Listener at localhost/41271] zookeeper.ZKUtil(164): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 15:00:16,991 DEBUG [Listener at localhost/41271] zookeeper.ZKUtil(164): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 15:00:16,992 DEBUG [Listener at localhost/41271] zookeeper.ZKUtil(164): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 15:00:16,995 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34479 2023-08-07 15:00:17,002 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34479 2023-08-07 15:00:17,009 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34479 2023-08-07 15:00:17,010 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34479 2023-08-07 15:00:17,010 DEBUG [Listener at localhost/41271] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34479 2023-08-07 15:00:17,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:17,014 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 15:00:17,014 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:17,017 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 15:00:17,017 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 15:00:17,017 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:17,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 15:00:17,018 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46343,1691420416931 from backup master directory 2023-08-07 15:00:17,018 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 15:00:17,020 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:17,020 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 15:00:17,020 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 15:00:17,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:17,060 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/hbase.id with ID: 2711d3fe-9617-4c7e-946a-0eccdd35c1a8 2023-08-07 15:00:17,074 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:17,077 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:17,089 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6cd0d7d0 to 127.0.0.1:60523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 15:00:17,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9725da5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 15:00:17,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 15:00:17,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 15:00:17,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 15:00:17,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store-tmp 2023-08-07 15:00:17,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 15:00:17,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 15:00:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:17,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:17,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 15:00:17,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:17,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 15:00:17,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/WALs/jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:17,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46343%2C1691420416931, suffix=, logDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/WALs/jenkins-hbase4.apache.org,46343,1691420416931, archiveDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/oldWALs, maxLogs=10 2023-08-07 15:00:17,137 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK] 2023-08-07 15:00:17,138 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK] 2023-08-07 15:00:17,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/WALs/jenkins-hbase4.apache.org,46343,1691420416931/jenkins-hbase4.apache.org%2C46343%2C1691420416931.1691420417120 2023-08-07 15:00:17,140 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK], DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK]] 2023-08-07 15:00:17,140 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 15:00:17,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 15:00:17,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 15:00:17,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 15:00:17,143 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 15:00:17,144 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 15:00:17,145 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 15:00:17,145 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:17,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 15:00:17,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 15:00:17,148 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 15:00:17,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 15:00:17,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=739250, jitterRate=-0.059996023774147034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 15:00:17,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 15:00:17,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 15:00:17,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 15:00:17,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 15:00:17,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 15:00:17,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 15:00:17,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-07 15:00:17,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 15:00:17,153 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-07 15:00:17,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-07 15:00:17,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 15:00:17,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 15:00:17,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-07 15:00:17,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 15:00:17,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-07 15:00:17,168 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:17,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-07 15:00:17,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-07 15:00:17,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-07 15:00:17,171 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 15:00:17,171 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 15:00:17,171 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:17,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46343,1691420416931, sessionid=0x101dff1de970000, setting cluster-up flag (Was=false) 2023-08-07 15:00:17,175 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:17,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-07 15:00:17,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:17,184 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:17,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-07 15:00:17,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:17,189 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/.hbase-snapshot/.tmp 2023-08-07 15:00:17,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-07 15:00:17,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 15:00:17,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 15:00:17,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 15:00:17,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 15:00:17,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 15:00:17,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 15:00:17,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691420447192 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 15:00:17,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,192 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 15:00:17,193 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-07 15:00:17,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 15:00:17,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 15:00:17,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 15:00:17,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 15:00:17,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 15:00:17,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420417193,5,FailOnTimeoutGroup] 2023-08-07 15:00:17,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420417193,5,FailOnTimeoutGroup] 2023-08-07 15:00:17,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 15:00:17,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,194 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 15:00:17,202 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 15:00:17,203 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 15:00:17,203 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547 2023-08-07 15:00:17,210 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 15:00:17,211 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 15:00:17,212 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(951): ClusterId : 2711d3fe-9617-4c7e-946a-0eccdd35c1a8 2023-08-07 15:00:17,213 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 15:00:17,213 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/info 2023-08-07 15:00:17,213 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 15:00:17,214 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:17,214 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 15:00:17,215 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/rep_barrier 2023-08-07 15:00:17,216 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 15:00:17,216 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 15:00:17,216 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 15:00:17,216 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:17,217 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 15:00:17,218 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 15:00:17,220 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/table 2023-08-07 15:00:17,220 DEBUG [RS:0;jenkins-hbase4:34479] zookeeper.ReadOnlyZKClient(139): Connect 0x32a120fb to 127.0.0.1:60523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 15:00:17,220 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 15:00:17,221 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:17,222 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740 2023-08-07 15:00:17,223 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740 2023-08-07 15:00:17,224 DEBUG [RS:0;jenkins-hbase4:34479] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c66c8df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 15:00:17,224 DEBUG [RS:0;jenkins-hbase4:34479] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38594b8b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 15:00:17,225 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 15:00:17,226 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 15:00:17,228 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 15:00:17,229 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=829193, jitterRate=0.054373741149902344}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 15:00:17,229 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 15:00:17,229 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 15:00:17,229 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 15:00:17,229 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 15:00:17,229 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 15:00:17,229 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 15:00:17,230 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 15:00:17,230 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 15:00:17,230 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 15:00:17,231 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-07 15:00:17,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 15:00:17,232 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 15:00:17,233 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 15:00:17,235 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34479 2023-08-07 15:00:17,235 INFO [RS:0;jenkins-hbase4:34479] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 15:00:17,235 INFO [RS:0;jenkins-hbase4:34479] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 15:00:17,235 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 15:00:17,236 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,46343,1691420416931 with isa=jenkins-hbase4.apache.org/172.31.14.131:34479, startcode=1691420416975 2023-08-07 15:00:17,236 DEBUG [RS:0;jenkins-hbase4:34479] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 15:00:17,244 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44035, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 15:00:17,245 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46343] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,245 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547 2023-08-07 15:00:17,246 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36247 2023-08-07 15:00:17,246 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 15:00:17,247 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 15:00:17,247 DEBUG [RS:0;jenkins-hbase4:34479] zookeeper.ZKUtil(162): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,247 WARN [RS:0;jenkins-hbase4:34479] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 15:00:17,247 INFO [RS:0;jenkins-hbase4:34479] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 15:00:17,248 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,250 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34479,1691420416975] 2023-08-07 15:00:17,253 DEBUG [RS:0;jenkins-hbase4:34479] zookeeper.ZKUtil(162): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,254 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-07 15:00:17,254 INFO [RS:0;jenkins-hbase4:34479] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 15:00:17,255 INFO [RS:0;jenkins-hbase4:34479] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 15:00:17,258 INFO [RS:0;jenkins-hbase4:34479] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 15:00:17,259 INFO [RS:0;jenkins-hbase4:34479] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,259 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 15:00:17,260 INFO [RS:0;jenkins-hbase4:34479] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,260 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,261 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,261 DEBUG [RS:0;jenkins-hbase4:34479] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 15:00:17,261 INFO [RS:0;jenkins-hbase4:34479] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,262 INFO [RS:0;jenkins-hbase4:34479] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,262 INFO [RS:0;jenkins-hbase4:34479] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,273 INFO [RS:0;jenkins-hbase4:34479] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 15:00:17,273 INFO [RS:0;jenkins-hbase4:34479] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34479,1691420416975-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,283 INFO [RS:0;jenkins-hbase4:34479] regionserver.Replication(203): jenkins-hbase4.apache.org,34479,1691420416975 started 2023-08-07 15:00:17,284 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34479,1691420416975, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34479, sessionid=0x101dff1de970001 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34479,1691420416975' 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34479,1691420416975' 2023-08-07 15:00:17,284 DEBUG [RS:0;jenkins-hbase4:34479] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-07 15:00:17,285 DEBUG [RS:0;jenkins-hbase4:34479] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-07 15:00:17,285 DEBUG [RS:0;jenkins-hbase4:34479] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 15:00:17,285 INFO [RS:0;jenkins-hbase4:34479] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 15:00:17,285 INFO [RS:0;jenkins-hbase4:34479] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 15:00:17,291 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 15:00:17,383 DEBUG [jenkins-hbase4:46343] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 15:00:17,384 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34479,1691420416975, state=OPENING 2023-08-07 15:00:17,386 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-07 15:00:17,387 INFO [RS:0;jenkins-hbase4:34479] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34479%2C1691420416975, suffix=, logDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/jenkins-hbase4.apache.org,34479,1691420416975, archiveDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/oldWALs, maxLogs=32 2023-08-07 15:00:17,387 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:17,388 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 15:00:17,388 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34479,1691420416975}] 2023-08-07 15:00:17,412 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK] 2023-08-07 15:00:17,412 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK] 2023-08-07 15:00:17,414 INFO [RS:0;jenkins-hbase4:34479] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/jenkins-hbase4.apache.org,34479,1691420416975/jenkins-hbase4.apache.org%2C34479%2C1691420416975.1691420417388 2023-08-07 15:00:17,415 DEBUG [RS:0;jenkins-hbase4:34479] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK], DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK]] 2023-08-07 15:00:17,542 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,542 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 15:00:17,544 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42940, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 15:00:17,548 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 15:00:17,548 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 15:00:17,550 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34479%2C1691420416975.meta, suffix=.meta, logDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/jenkins-hbase4.apache.org,34479,1691420416975, archiveDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/oldWALs, maxLogs=32 2023-08-07 15:00:17,570 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK] 2023-08-07 15:00:17,571 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK] 2023-08-07 15:00:17,611 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/jenkins-hbase4.apache.org,34479,1691420416975/jenkins-hbase4.apache.org%2C34479%2C1691420416975.meta.1691420417551.meta 2023-08-07 15:00:17,611 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK], DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK]] 2023-08-07 15:00:17,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 15:00:17,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 15:00:17,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 15:00:17,612 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 15:00:17,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 15:00:17,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 15:00:17,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 15:00:17,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 15:00:17,617 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 15:00:17,618 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/info 2023-08-07 15:00:17,619 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/info 2023-08-07 15:00:17,619 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 15:00:17,620 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:17,620 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 15:00:17,622 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/rep_barrier 2023-08-07 15:00:17,622 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/rep_barrier 2023-08-07 15:00:17,623 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 15:00:17,623 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:17,624 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 15:00:17,625 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/table 2023-08-07 15:00:17,625 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/table 2023-08-07 15:00:17,625 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 15:00:17,626 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:17,628 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740 2023-08-07 15:00:17,630 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740 2023-08-07 15:00:17,634 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-07 15:00:17,636 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 15:00:17,640 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=736649, jitterRate=-0.06330245733261108}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-07 15:00:17,640 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 15:00:17,641 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691420417542 2023-08-07 15:00:17,646 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 15:00:17,647 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 15:00:17,647 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34479,1691420416975, state=OPEN 2023-08-07 15:00:17,649 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-07 15:00:17,649 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 15:00:17,651 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-07 15:00:17,651 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34479,1691420416975 in 261 msec 2023-08-07 15:00:17,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-07 15:00:17,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 420 msec 2023-08-07 15:00:17,658 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 467 msec 2023-08-07 15:00:17,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691420417658, completionTime=-1 2023-08-07 15:00:17,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-07 15:00:17,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 15:00:17,663 DEBUG [hconnection-0x1891e08d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 15:00:17,666 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42952, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 15:00:17,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-07 15:00:17,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691420477668 2023-08-07 15:00:17,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691420537668 2023-08-07 15:00:17,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 9 msec 2023-08-07 15:00:17,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46343,1691420416931-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46343,1691420416931-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46343,1691420416931-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46343, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 15:00:17,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-07 15:00:17,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 15:00:17,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-07 15:00:17,677 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-07 15:00:17,679 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 15:00:17,680 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 15:00:17,682 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/.tmp/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:17,683 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/.tmp/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72 empty. 2023-08-07 15:00:17,684 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/.tmp/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:17,684 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-07 15:00:17,700 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-07 15:00:17,701 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 48aaf63db78e76ee9bb78449594eca72, NAME => 'hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/.tmp 2023-08-07 15:00:17,709 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 15:00:17,709 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 48aaf63db78e76ee9bb78449594eca72, disabling compactions & flushes 2023-08-07 15:00:17,709 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:17,709 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:17,709 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. after waiting 0 ms 2023-08-07 15:00:17,709 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:17,709 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:17,709 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 48aaf63db78e76ee9bb78449594eca72: 2023-08-07 15:00:17,711 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 15:00:17,712 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420417712"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691420417712"}]},"ts":"1691420417712"} 2023-08-07 15:00:17,714 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 15:00:17,714 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 15:00:17,714 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420417714"}]},"ts":"1691420417714"} 2023-08-07 15:00:17,715 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-07 15:00:17,721 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=48aaf63db78e76ee9bb78449594eca72, ASSIGN}] 2023-08-07 15:00:17,723 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=48aaf63db78e76ee9bb78449594eca72, ASSIGN 2023-08-07 15:00:17,724 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=48aaf63db78e76ee9bb78449594eca72, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34479,1691420416975; forceNewPlan=false, retain=false 2023-08-07 15:00:17,875 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=48aaf63db78e76ee9bb78449594eca72, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:17,875 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420417875"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691420417875"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691420417875"}]},"ts":"1691420417875"} 2023-08-07 15:00:17,877 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 48aaf63db78e76ee9bb78449594eca72, server=jenkins-hbase4.apache.org,34479,1691420416975}] 2023-08-07 15:00:18,032 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 48aaf63db78e76ee9bb78449594eca72, NAME => 'hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72.', STARTKEY => '', ENDKEY => ''} 2023-08-07 15:00:18,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 15:00:18,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,034 INFO [StoreOpener-48aaf63db78e76ee9bb78449594eca72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,036 DEBUG [StoreOpener-48aaf63db78e76ee9bb78449594eca72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/info 2023-08-07 15:00:18,036 DEBUG [StoreOpener-48aaf63db78e76ee9bb78449594eca72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/info 2023-08-07 15:00:18,037 INFO [StoreOpener-48aaf63db78e76ee9bb78449594eca72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 48aaf63db78e76ee9bb78449594eca72 columnFamilyName info 2023-08-07 15:00:18,037 INFO [StoreOpener-48aaf63db78e76ee9bb78449594eca72-1] regionserver.HStore(310): Store=48aaf63db78e76ee9bb78449594eca72/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 15:00:18,038 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,039 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,042 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,047 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 15:00:18,047 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 48aaf63db78e76ee9bb78449594eca72; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=801530, jitterRate=0.019199103116989136}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 15:00:18,048 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 48aaf63db78e76ee9bb78449594eca72: 2023-08-07 15:00:18,048 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72., pid=6, masterSystemTime=1691420418028 2023-08-07 15:00:18,050 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,050 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,051 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=48aaf63db78e76ee9bb78449594eca72, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:18,051 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691420418051"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691420418051"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691420418051"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691420418051"}]},"ts":"1691420418051"} 2023-08-07 15:00:18,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-07 15:00:18,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 48aaf63db78e76ee9bb78449594eca72, server=jenkins-hbase4.apache.org,34479,1691420416975 in 175 msec 2023-08-07 15:00:18,058 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-07 15:00:18,059 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=48aaf63db78e76ee9bb78449594eca72, ASSIGN in 335 msec 2023-08-07 15:00:18,059 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 15:00:18,060 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691420418060"}]},"ts":"1691420418060"} 2023-08-07 15:00:18,061 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-07 15:00:18,063 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 15:00:18,066 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 389 msec 2023-08-07 15:00:18,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-07 15:00:18,080 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-07 15:00:18,080 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:18,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-07 15:00:18,096 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 15:00:18,099 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 13 msec 2023-08-07 15:00:18,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-07 15:00:18,114 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 15:00:18,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 10 msec 2023-08-07 15:00:18,132 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-07 15:00:18,138 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-07 15:00:18,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.119sec 2023-08-07 15:00:18,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 15:00:18,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 15:00:18,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 15:00:18,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46343,1691420416931-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 15:00:18,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46343,1691420416931-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 15:00:18,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 15:00:18,213 DEBUG [Listener at localhost/41271] zookeeper.ReadOnlyZKClient(139): Connect 0x3e7ac4bc to 127.0.0.1:60523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 15:00:18,219 DEBUG [Listener at localhost/41271] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76d08e8b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 15:00:18,221 DEBUG [hconnection-0x2660b5a7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 15:00:18,223 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42966, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 15:00:18,224 INFO [Listener at localhost/41271] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:18,224 INFO [Listener at localhost/41271] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 15:00:18,228 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-07 15:00:18,228 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:18,230 INFO [Listener at localhost/41271] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-07 15:00:18,231 INFO [Listener at localhost/41271] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 15:00:18,234 INFO [Listener at localhost/41271] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/test.com,8080,1, archiveDir=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/oldWALs, maxLogs=32 2023-08-07 15:00:18,254 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK] 2023-08-07 15:00:18,255 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK] 2023-08-07 15:00:18,263 INFO [Listener at localhost/41271] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/test.com,8080,1/test.com%2C8080%2C1.1691420418235 2023-08-07 15:00:18,266 DEBUG [Listener at localhost/41271] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK], DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK]] 2023-08-07 15:00:18,290 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK] 2023-08-07 15:00:18,290 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK] 2023-08-07 15:00:18,295 INFO [Listener at localhost/41271] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/test.com,8080,1/test.com%2C8080%2C1.1691420418235 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/test.com,8080,1/test.com%2C8080%2C1.1691420418267 2023-08-07 15:00:18,296 DEBUG [Listener at localhost/41271] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33099,DS-fde23b82-624a-473b-a0ea-70024d38836d,DISK], DatanodeInfoWithStorage[127.0.0.1:45241,DS-0f12b1fc-2969-4155-a7f6-eb613dfb31eb,DISK]] 2023-08-07 15:00:18,296 DEBUG [Listener at localhost/41271] wal.AbstractFSWAL(716): hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/test.com,8080,1/test.com%2C8080%2C1.1691420418235 is not closed yet, will try archiving it next time 2023-08-07 15:00:18,312 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/test.com,8080,1/test.com%2C8080%2C1.1691420418235 to hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/oldWALs/test.com%2C8080%2C1.1691420418235 2023-08-07 15:00:18,316 DEBUG [Listener at localhost/41271] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/oldWALs 2023-08-07 15:00:18,316 INFO [Listener at localhost/41271] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1691420418267) 2023-08-07 15:00:18,316 INFO [Listener at localhost/41271] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-07 15:00:18,317 DEBUG [Listener at localhost/41271] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e7ac4bc to 127.0.0.1:60523 2023-08-07 15:00:18,317 DEBUG [Listener at localhost/41271] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:18,318 DEBUG [Listener at localhost/41271] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 15:00:18,318 DEBUG [Listener at localhost/41271] util.JVMClusterUtil(257): Found active master hash=1533916583, stopped=false 2023-08-07 15:00:18,318 INFO [Listener at localhost/41271] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:18,326 INFO [Listener at localhost/41271] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 15:00:18,326 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 15:00:18,326 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 15:00:18,327 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:18,327 DEBUG [Listener at localhost/41271] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6cd0d7d0 to 127.0.0.1:60523 2023-08-07 15:00:18,328 DEBUG [Listener at localhost/41271] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:18,328 INFO [Listener at localhost/41271] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34479,1691420416975' ***** 2023-08-07 15:00:18,328 INFO [Listener at localhost/41271] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 15:00:18,328 INFO [RS:0;jenkins-hbase4:34479] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 15:00:18,328 INFO [RS:0;jenkins-hbase4:34479] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 15:00:18,328 INFO [RS:0;jenkins-hbase4:34479] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 15:00:18,328 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(3305): Received CLOSE for 48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,328 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 15:00:18,330 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:18,331 DEBUG [RS:0;jenkins-hbase4:34479] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32a120fb to 127.0.0.1:60523 2023-08-07 15:00:18,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 15:00:18,331 DEBUG [RS:0;jenkins-hbase4:34479] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:18,336 INFO [RS:0;jenkins-hbase4:34479] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 15:00:18,336 INFO [RS:0;jenkins-hbase4:34479] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 15:00:18,336 INFO [RS:0;jenkins-hbase4:34479] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 15:00:18,336 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 15:00:18,337 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 15:00:18,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 48aaf63db78e76ee9bb78449594eca72, disabling compactions & flushes 2023-08-07 15:00:18,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. after waiting 0 ms 2023-08-07 15:00:18,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 48aaf63db78e76ee9bb78449594eca72 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-07 15:00:18,342 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-07 15:00:18,342 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1478): Online Regions={48aaf63db78e76ee9bb78449594eca72=hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72., 1588230740=hbase:meta,,1.1588230740} 2023-08-07 15:00:18,344 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1504): Waiting on 1588230740, 48aaf63db78e76ee9bb78449594eca72 2023-08-07 15:00:18,344 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 15:00:18,344 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 15:00:18,344 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 15:00:18,344 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 15:00:18,344 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 15:00:18,344 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-08-07 15:00:18,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/.tmp/info/3a083b567404481fae566e4e694b7f14 2023-08-07 15:00:18,387 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/.tmp/info/7d487ef1aaa14b6c9d2f583675c89c14 2023-08-07 15:00:18,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/.tmp/info/3a083b567404481fae566e4e694b7f14 as hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/info/3a083b567404481fae566e4e694b7f14 2023-08-07 15:00:18,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/info/3a083b567404481fae566e4e694b7f14, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 15:00:18,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 48aaf63db78e76ee9bb78449594eca72 in 86ms, sequenceid=6, compaction requested=false 2023-08-07 15:00:18,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/namespace/48aaf63db78e76ee9bb78449594eca72/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 15:00:18,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 48aaf63db78e76ee9bb78449594eca72: 2023-08-07 15:00:18,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691420417675.48aaf63db78e76ee9bb78449594eca72. 2023-08-07 15:00:18,544 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-07 15:00:18,744 DEBUG [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-07 15:00:18,836 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/.tmp/table/8b32204ab849499b947bcd53b39cb047 2023-08-07 15:00:18,842 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/.tmp/info/7d487ef1aaa14b6c9d2f583675c89c14 as hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/info/7d487ef1aaa14b6c9d2f583675c89c14 2023-08-07 15:00:18,848 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/info/7d487ef1aaa14b6c9d2f583675c89c14, entries=10, sequenceid=9, filesize=5.9 K 2023-08-07 15:00:18,849 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/.tmp/table/8b32204ab849499b947bcd53b39cb047 as hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/table/8b32204ab849499b947bcd53b39cb047 2023-08-07 15:00:18,854 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/table/8b32204ab849499b947bcd53b39cb047, entries=2, sequenceid=9, filesize=4.7 K 2023-08-07 15:00:18,855 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 511ms, sequenceid=9, compaction requested=false 2023-08-07 15:00:18,872 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-08-07 15:00:18,873 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 15:00:18,873 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 15:00:18,873 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 15:00:18,873 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 15:00:18,944 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34479,1691420416975; all regions closed. 2023-08-07 15:00:18,949 DEBUG [RS:0;jenkins-hbase4:34479] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/oldWALs 2023-08-07 15:00:18,949 INFO [RS:0;jenkins-hbase4:34479] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34479%2C1691420416975.meta:.meta(num 1691420417551) 2023-08-07 15:00:18,953 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/WALs/jenkins-hbase4.apache.org,34479,1691420416975/jenkins-hbase4.apache.org%2C34479%2C1691420416975.1691420417388 not finished, retry = 0 2023-08-07 15:00:19,055 DEBUG [RS:0;jenkins-hbase4:34479] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/oldWALs 2023-08-07 15:00:19,055 INFO [RS:0;jenkins-hbase4:34479] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34479%2C1691420416975:(num 1691420417388) 2023-08-07 15:00:19,056 DEBUG [RS:0;jenkins-hbase4:34479] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:19,056 INFO [RS:0;jenkins-hbase4:34479] regionserver.LeaseManager(133): Closed leases 2023-08-07 15:00:19,056 INFO [RS:0;jenkins-hbase4:34479] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 15:00:19,056 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 15:00:19,057 INFO [RS:0;jenkins-hbase4:34479] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34479 2023-08-07 15:00:19,060 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 15:00:19,060 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34479,1691420416975 2023-08-07 15:00:19,060 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 15:00:19,061 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34479,1691420416975] 2023-08-07 15:00:19,061 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34479,1691420416975; numProcessing=1 2023-08-07 15:00:19,063 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34479,1691420416975 already deleted, retry=false 2023-08-07 15:00:19,063 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34479,1691420416975 expired; onlineServers=0 2023-08-07 15:00:19,063 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46343,1691420416931' ***** 2023-08-07 15:00:19,063 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 15:00:19,063 DEBUG [M:0;jenkins-hbase4:46343] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f523a12, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 15:00:19,063 INFO [M:0;jenkins-hbase4:46343] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46343,1691420416931 2023-08-07 15:00:19,063 INFO [M:0;jenkins-hbase4:46343] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46343,1691420416931; all regions closed. 2023-08-07 15:00:19,064 DEBUG [M:0;jenkins-hbase4:46343] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 15:00:19,064 DEBUG [M:0;jenkins-hbase4:46343] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 15:00:19,064 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 15:00:19,064 DEBUG [M:0;jenkins-hbase4:46343] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 15:00:19,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420417193] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691420417193,5,FailOnTimeoutGroup] 2023-08-07 15:00:19,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420417193] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691420417193,5,FailOnTimeoutGroup] 2023-08-07 15:00:19,065 INFO [M:0;jenkins-hbase4:46343] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 15:00:19,065 INFO [M:0;jenkins-hbase4:46343] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 15:00:19,065 INFO [M:0;jenkins-hbase4:46343] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 15:00:19,066 DEBUG [M:0;jenkins-hbase4:46343] master.HMaster(1512): Stopping service threads 2023-08-07 15:00:19,066 INFO [M:0;jenkins-hbase4:46343] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 15:00:19,066 ERROR [M:0;jenkins-hbase4:46343] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-08-07 15:00:19,066 INFO [M:0;jenkins-hbase4:46343] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 15:00:19,066 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 15:00:19,066 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-07 15:00:19,066 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 15:00:19,066 DEBUG [M:0;jenkins-hbase4:46343] zookeeper.ZKUtil(398): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-07 15:00:19,067 WARN [M:0;jenkins-hbase4:46343] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 15:00:19,067 INFO [M:0;jenkins-hbase4:46343] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 15:00:19,067 INFO [M:0;jenkins-hbase4:46343] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 15:00:19,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 15:00:19,067 DEBUG [M:0;jenkins-hbase4:46343] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 15:00:19,067 INFO [M:0;jenkins-hbase4:46343] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:19,067 DEBUG [M:0;jenkins-hbase4:46343] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:19,067 DEBUG [M:0;jenkins-hbase4:46343] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 15:00:19,067 DEBUG [M:0;jenkins-hbase4:46343] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:19,067 INFO [M:0;jenkins-hbase4:46343] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-08-07 15:00:19,077 INFO [M:0;jenkins-hbase4:46343] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e56efcffd97e4bb699819c5ff8e71e33 2023-08-07 15:00:19,082 DEBUG [M:0;jenkins-hbase4:46343] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e56efcffd97e4bb699819c5ff8e71e33 as hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e56efcffd97e4bb699819c5ff8e71e33 2023-08-07 15:00:19,086 INFO [M:0;jenkins-hbase4:46343] regionserver.HStore(1080): Added hdfs://localhost:36247/user/jenkins/test-data/1db2d9f6-329d-8b4a-07a2-094e54f24547/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e56efcffd97e4bb699819c5ff8e71e33, entries=8, sequenceid=66, filesize=6.3 K 2023-08-07 15:00:19,087 INFO [M:0;jenkins-hbase4:46343] regionserver.HRegion(2948): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=66, compaction requested=false 2023-08-07 15:00:19,088 INFO [M:0;jenkins-hbase4:46343] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 15:00:19,088 DEBUG [M:0;jenkins-hbase4:46343] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 15:00:19,091 INFO [M:0;jenkins-hbase4:46343] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 15:00:19,091 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 15:00:19,092 INFO [M:0;jenkins-hbase4:46343] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46343 2023-08-07 15:00:19,094 DEBUG [M:0;jenkins-hbase4:46343] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46343,1691420416931 already deleted, retry=false 2023-08-07 15:00:19,162 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:19,162 INFO [RS:0;jenkins-hbase4:34479] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34479,1691420416975; zookeeper connection closed. 2023-08-07 15:00:19,162 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): regionserver:34479-0x101dff1de970001, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:19,163 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@610dddcd] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@610dddcd 2023-08-07 15:00:19,163 INFO [Listener at localhost/41271] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-07 15:00:19,262 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:19,262 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 15:00:19,262 DEBUG [Listener at localhost/41271-EventThread] zookeeper.ZKWatcher(600): master:46343-0x101dff1de970000, quorum=127.0.0.1:60523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 15:00:19,262 INFO [M:0;jenkins-hbase4:46343] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46343,1691420416931; zookeeper connection closed. 2023-08-07 15:00:19,265 WARN [Listener at localhost/41271] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 15:00:19,268 INFO [Listener at localhost/41271] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 15:00:19,372 WARN [BP-267598421-172.31.14.131-1691420416246 heartbeating to localhost/127.0.0.1:36247] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 15:00:19,372 WARN [BP-267598421-172.31.14.131-1691420416246 heartbeating to localhost/127.0.0.1:36247] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-267598421-172.31.14.131-1691420416246 (Datanode Uuid 46af531b-4b0b-4670-bca4-770962143850) service to localhost/127.0.0.1:36247 2023-08-07 15:00:19,372 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305/dfs/data/data3/current/BP-267598421-172.31.14.131-1691420416246] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:19,373 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305/dfs/data/data4/current/BP-267598421-172.31.14.131-1691420416246] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:19,373 WARN [Listener at localhost/41271] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 15:00:19,378 INFO [Listener at localhost/41271] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 15:00:19,481 WARN [BP-267598421-172.31.14.131-1691420416246 heartbeating to localhost/127.0.0.1:36247] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 15:00:19,481 WARN [BP-267598421-172.31.14.131-1691420416246 heartbeating to localhost/127.0.0.1:36247] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-267598421-172.31.14.131-1691420416246 (Datanode Uuid 8263d490-5de5-4462-a84f-1a6d46cdf892) service to localhost/127.0.0.1:36247 2023-08-07 15:00:19,482 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305/dfs/data/data1/current/BP-267598421-172.31.14.131-1691420416246] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:19,482 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/05a7889b-9e07-27d3-4020-119974bc85c4/cluster_a7c83895-c74a-e028-accd-47d7fb3df305/dfs/data/data2/current/BP-267598421-172.31.14.131-1691420416246] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 15:00:19,502 INFO [Listener at localhost/41271] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 15:00:19,613 INFO [Listener at localhost/41271] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-07 15:00:19,630 INFO [Listener at localhost/41271] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-07 15:00:19,643 INFO [Listener at localhost/41271] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=99 (was 77) - Thread LEAK? -, OpenFileDescriptor=494 (was 467) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=220 (was 220), ProcessCount=171 (was 171), AvailableMemoryMB=1348 (was 1498)