2023-02-11 09:53:56,038 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-11 09:53:56,287 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9 2023-02-11 09:53:56,301 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-11 09:53:56,302 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7, deleteOnExit=true 2023-02-11 09:53:56,302 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-11 09:53:56,302 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/test.cache.data in system properties and HBase conf 2023-02-11 09:53:56,303 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/hadoop.tmp.dir in system properties and HBase conf 2023-02-11 09:53:56,303 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/hadoop.log.dir in system properties and HBase conf 2023-02-11 09:53:56,304 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-11 09:53:56,304 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-11 09:53:56,304 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-11 09:53:56,410 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-11 09:53:56,784 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-11 09:53:56,788 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-11 09:53:56,788 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-11 09:53:56,789 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-11 09:53:56,789 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 09:53:56,789 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-11 09:53:56,790 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-11 09:53:56,790 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 09:53:56,790 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 09:53:56,791 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-11 09:53:56,791 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/nfs.dump.dir in system properties and HBase conf 2023-02-11 09:53:56,791 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/java.io.tmpdir in system properties and HBase conf 2023-02-11 09:53:56,792 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 09:53:56,792 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-11 09:53:56,792 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-11 09:53:57,312 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 09:53:57,316 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 09:53:57,591 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-11 09:53:57,738 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-11 09:53:57,753 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 09:53:57,789 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 09:53:57,824 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/java.io.tmpdir/Jetty_localhost_36195_hdfs____.45fde4/webapp 2023-02-11 09:53:57,986 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36195 2023-02-11 09:53:57,996 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 09:53:57,996 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 09:53:58,387 WARN [Listener at localhost/41137] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 09:53:58,495 WARN [Listener at localhost/41137] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 09:53:58,512 WARN [Listener at localhost/41137] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 09:53:58,518 INFO [Listener at localhost/41137] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 09:53:58,523 INFO [Listener at localhost/41137] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/java.io.tmpdir/Jetty_localhost_44473_datanode____60e2yf/webapp 2023-02-11 09:53:58,625 INFO [Listener at localhost/41137] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44473 2023-02-11 09:53:58,939 WARN [Listener at localhost/41943] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 09:53:58,950 WARN [Listener at localhost/41943] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 09:53:58,956 WARN [Listener at localhost/41943] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 09:53:58,958 INFO [Listener at localhost/41943] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 09:53:58,964 INFO [Listener at localhost/41943] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/java.io.tmpdir/Jetty_localhost_33895_datanode____.5nmckl/webapp 2023-02-11 09:53:59,089 INFO [Listener at localhost/41943] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33895 2023-02-11 09:53:59,098 WARN [Listener at localhost/40955] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 09:53:59,111 WARN [Listener at localhost/40955] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 09:53:59,115 WARN [Listener at localhost/40955] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 09:53:59,117 INFO [Listener at localhost/40955] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 09:53:59,126 INFO [Listener at localhost/40955] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/java.io.tmpdir/Jetty_localhost_39741_datanode____.k17knf/webapp 2023-02-11 09:53:59,252 INFO [Listener at localhost/40955] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39741 2023-02-11 09:53:59,264 WARN [Listener at localhost/34597] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 09:53:59,411 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8ee46f10ae300f44: Processing first storage report for DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0 from datanode 6b5fb81e-4ae9-4619-8579-f3d9aa9248cc 2023-02-11 09:53:59,413 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8ee46f10ae300f44: from storage DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0 node DatanodeRegistration(127.0.0.1:36925, datanodeUuid=6b5fb81e-4ae9-4619-8579-f3d9aa9248cc, infoPort=38479, infoSecurePort=0, ipcPort=34597, storageInfo=lv=-57;cid=testClusterID;nsid=396835378;c=1676109237390), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-11 09:53:59,413 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3a090f8520157109: Processing first storage report for DS-3e692238-0b4a-4eaa-997e-4c3ec947668d from datanode b979c166-8ceb-4cd3-99a3-1b2b2b1e0014 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3a090f8520157109: from storage DS-3e692238-0b4a-4eaa-997e-4c3ec947668d node DatanodeRegistration(127.0.0.1:37777, datanodeUuid=b979c166-8ceb-4cd3-99a3-1b2b2b1e0014, infoPort=39839, infoSecurePort=0, ipcPort=41943, storageInfo=lv=-57;cid=testClusterID;nsid=396835378;c=1676109237390), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbb3c9f4906dc3d8c: Processing first storage report for DS-61d366ad-15e1-4269-bcaa-811227d6beff from datanode 75b11f66-72e0-4ff6-b627-e436c09781c9 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbb3c9f4906dc3d8c: from storage DS-61d366ad-15e1-4269-bcaa-811227d6beff node DatanodeRegistration(127.0.0.1:43349, datanodeUuid=75b11f66-72e0-4ff6-b627-e436c09781c9, infoPort=35041, infoSecurePort=0, ipcPort=40955, storageInfo=lv=-57;cid=testClusterID;nsid=396835378;c=1676109237390), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3a090f8520157109: Processing first storage report for DS-b7943c96-59d6-449d-bf4d-97e57ca64d55 from datanode b979c166-8ceb-4cd3-99a3-1b2b2b1e0014 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3a090f8520157109: from storage DS-b7943c96-59d6-449d-bf4d-97e57ca64d55 node DatanodeRegistration(127.0.0.1:37777, datanodeUuid=b979c166-8ceb-4cd3-99a3-1b2b2b1e0014, infoPort=39839, infoSecurePort=0, ipcPort=41943, storageInfo=lv=-57;cid=testClusterID;nsid=396835378;c=1676109237390), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8ee46f10ae300f44: Processing first storage report for DS-b78e8cea-6878-4bb9-a017-d3831f171e72 from datanode 6b5fb81e-4ae9-4619-8579-f3d9aa9248cc 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8ee46f10ae300f44: from storage DS-b78e8cea-6878-4bb9-a017-d3831f171e72 node DatanodeRegistration(127.0.0.1:36925, datanodeUuid=6b5fb81e-4ae9-4619-8579-f3d9aa9248cc, infoPort=38479, infoSecurePort=0, ipcPort=34597, storageInfo=lv=-57;cid=testClusterID;nsid=396835378;c=1676109237390), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbb3c9f4906dc3d8c: Processing first storage report for DS-3c3f1d33-d50e-4646-bea3-367a9cd8cfcb from datanode 75b11f66-72e0-4ff6-b627-e436c09781c9 2023-02-11 09:53:59,414 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbb3c9f4906dc3d8c: from storage DS-3c3f1d33-d50e-4646-bea3-367a9cd8cfcb node DatanodeRegistration(127.0.0.1:43349, datanodeUuid=75b11f66-72e0-4ff6-b627-e436c09781c9, infoPort=35041, infoSecurePort=0, ipcPort=40955, storageInfo=lv=-57;cid=testClusterID;nsid=396835378;c=1676109237390), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 09:53:59,640 DEBUG [Listener at localhost/34597] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9 2023-02-11 09:53:59,708 INFO [Listener at localhost/34597] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/zookeeper_0, clientPort=49784, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-11 09:53:59,725 INFO [Listener at localhost/34597] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=49784 2023-02-11 09:53:59,734 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:53:59,736 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:00,416 INFO [Listener at localhost/34597] util.FSUtils(479): Created version file at hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16 with version=8 2023-02-11 09:54:00,416 INFO [Listener at localhost/34597] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/hbase-staging 2023-02-11 09:54:00,713 INFO [Listener at localhost/34597] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-11 09:54:01,171 INFO [Listener at localhost/34597] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 09:54:01,203 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,204 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,205 INFO [Listener at localhost/34597] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 09:54:01,205 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,205 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 09:54:01,351 INFO [Listener at localhost/34597] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 09:54:01,430 DEBUG [Listener at localhost/34597] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-11 09:54:01,521 INFO [Listener at localhost/34597] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43041 2023-02-11 09:54:01,533 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,535 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,555 INFO [Listener at localhost/34597] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43041 connecting to ZooKeeper ensemble=127.0.0.1:49784 2023-02-11 09:54:01,595 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:430410x0, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 09:54:01,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:43041-0x10042cd3f510000 connected 2023-02-11 09:54:01,619 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 09:54:01,620 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:54:01,623 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 09:54:01,631 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43041 2023-02-11 09:54:01,631 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43041 2023-02-11 09:54:01,631 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43041 2023-02-11 09:54:01,632 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43041 2023-02-11 09:54:01,632 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43041 2023-02-11 09:54:01,638 INFO [Listener at localhost/34597] master.HMaster(439): hbase.rootdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16, hbase.cluster.distributed=false 2023-02-11 09:54:01,701 INFO [Listener at localhost/34597] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 09:54:01,702 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,702 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,702 INFO [Listener at localhost/34597] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 09:54:01,702 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,702 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 09:54:01,707 INFO [Listener at localhost/34597] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 09:54:01,709 INFO [Listener at localhost/34597] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40095 2023-02-11 09:54:01,711 INFO [Listener at localhost/34597] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 09:54:01,717 DEBUG [Listener at localhost/34597] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 09:54:01,718 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,720 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,722 INFO [Listener at localhost/34597] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40095 connecting to ZooKeeper ensemble=127.0.0.1:49784 2023-02-11 09:54:01,728 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:400950x0, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 09:54:01,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40095-0x10042cd3f510001 connected 2023-02-11 09:54:01,729 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 09:54:01,730 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:54:01,731 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 09:54:01,731 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40095 2023-02-11 09:54:01,732 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40095 2023-02-11 09:54:01,732 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40095 2023-02-11 09:54:01,733 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40095 2023-02-11 09:54:01,733 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40095 2023-02-11 09:54:01,745 INFO [Listener at localhost/34597] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 09:54:01,746 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,746 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,746 INFO [Listener at localhost/34597] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 09:54:01,746 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,747 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 09:54:01,747 INFO [Listener at localhost/34597] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 09:54:01,748 INFO [Listener at localhost/34597] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34799 2023-02-11 09:54:01,749 INFO [Listener at localhost/34597] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 09:54:01,750 DEBUG [Listener at localhost/34597] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 09:54:01,750 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,753 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,756 INFO [Listener at localhost/34597] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34799 connecting to ZooKeeper ensemble=127.0.0.1:49784 2023-02-11 09:54:01,759 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:347990x0, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 09:54:01,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34799-0x10042cd3f510002 connected 2023-02-11 09:54:01,761 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 09:54:01,761 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:54:01,762 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 09:54:01,763 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34799 2023-02-11 09:54:01,763 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34799 2023-02-11 09:54:01,763 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34799 2023-02-11 09:54:01,764 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34799 2023-02-11 09:54:01,764 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34799 2023-02-11 09:54:01,776 INFO [Listener at localhost/34597] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 09:54:01,776 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,777 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,777 INFO [Listener at localhost/34597] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 09:54:01,777 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 09:54:01,777 INFO [Listener at localhost/34597] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 09:54:01,777 INFO [Listener at localhost/34597] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 09:54:01,778 INFO [Listener at localhost/34597] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41949 2023-02-11 09:54:01,779 INFO [Listener at localhost/34597] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 09:54:01,780 DEBUG [Listener at localhost/34597] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 09:54:01,781 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,783 INFO [Listener at localhost/34597] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,784 INFO [Listener at localhost/34597] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41949 connecting to ZooKeeper ensemble=127.0.0.1:49784 2023-02-11 09:54:01,787 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:419490x0, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 09:54:01,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41949-0x10042cd3f510003 connected 2023-02-11 09:54:01,789 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 09:54:01,789 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:54:01,790 DEBUG [Listener at localhost/34597] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 09:54:01,791 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41949 2023-02-11 09:54:01,791 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41949 2023-02-11 09:54:01,791 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41949 2023-02-11 09:54:01,792 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41949 2023-02-11 09:54:01,792 DEBUG [Listener at localhost/34597] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41949 2023-02-11 09:54:01,793 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:01,802 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 09:54:01,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:01,823 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 09:54:01,823 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 09:54:01,823 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 09:54:01,823 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 09:54:01,823 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:01,825 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 09:54:01,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43041,1676109240552 from backup master directory 2023-02-11 09:54:01,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 09:54:01,830 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:01,830 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 09:54:01,831 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 09:54:01,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:01,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-11 09:54:01,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-11 09:54:01,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/hbase.id with ID: 0b952cb4-2365-4b3a-baca-c242752113b4 2023-02-11 09:54:01,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 09:54:01,983 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:02,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x03edf9f1 to 127.0.0.1:49784 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 09:54:02,065 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5d8f4959, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 09:54:02,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 09:54:02,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-11 09:54:02,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-11 09:54:02,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-11 09:54:02,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:54:02,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:54:02,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 09:54:02,153 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store-tmp 2023-02-11 09:54:02,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:02,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 09:54:02,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:54:02,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:54:02,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 09:54:02,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:54:02,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:54:02,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676109242219Disabling compacts and flushes for region at 1676109242219Disabling writes for close at 1676109242220 (+1 ms)Writing region close event to WAL at 1676109242220Closed at 1676109242220 2023-02-11 09:54:02,221 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/WALs/jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:02,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43041%2C1676109240552, suffix=, logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/WALs/jenkins-hbase4.apache.org,43041,1676109240552, archiveDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/oldWALs, maxLogs=10 2023-02-11 09:54:02,293 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK] 2023-02-11 09:54:02,293 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK] 2023-02-11 09:54:02,293 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK] 2023-02-11 09:54:02,301 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:54:02,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/WALs/jenkins-hbase4.apache.org,43041,1676109240552/jenkins-hbase4.apache.org%2C43041%2C1676109240552.1676109242250 2023-02-11 09:54:02,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK], DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK], DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK]] 2023-02-11 09:54:02,368 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:02,368 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:02,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 09:54:02,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 09:54:02,427 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-11 09:54:02,434 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-11 09:54:02,457 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-11 09:54:02,468 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:02,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 09:54:02,475 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 09:54:02,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 09:54:02,493 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:02,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72438600, jitterRate=0.07941925525665283}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 09:54:02,494 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676109242397Initializing all the Stores at 1676109242399 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109242399Cleaning up temporary data from old regions at 1676109242481 (+82 ms)Cleaning up detritus from prior splits at 1676109242482 (+1 ms)Region opened successfully at 1676109242494 (+12 ms) 2023-02-11 09:54:02,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-11 09:54:02,514 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:02,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:02,518 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:02,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-11 09:54:02,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-02-11 09:54:02,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-11 09:54:02,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-11 09:54:02,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-11 09:54:02,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-11 09:54:02,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-11 09:54:02,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-11 09:54:02,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-11 09:54:02,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-11 09:54:02,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:02,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-11 09:54:02,621 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-11 09:54:02,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-11 09:54:02,637 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 09:54:02,637 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 09:54:02,637 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 09:54:02,637 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 09:54:02,638 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:02,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,43041,1676109240552, sessionid=0x10042cd3f510000, setting cluster-up flag (Was=false) 2023-02-11 09:54:02,657 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:02,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-11 09:54:02,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:02,670 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:02,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-11 09:54:02,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:02,680 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.hbase-snapshot/.tmp 2023-02-11 09:54:02,695 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(952): ClusterId : 0b952cb4-2365-4b3a-baca-c242752113b4 2023-02-11 09:54:02,695 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(952): ClusterId : 0b952cb4-2365-4b3a-baca-c242752113b4 2023-02-11 09:54:02,696 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(952): ClusterId : 0b952cb4-2365-4b3a-baca-c242752113b4 2023-02-11 09:54:02,701 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 09:54:02,701 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 09:54:02,701 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 09:54:02,707 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 09:54:02,707 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 09:54:02,707 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 09:54:02,707 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 09:54:02,707 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 09:54:02,707 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 09:54:02,712 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 09:54:02,713 DEBUG [RS:2;jenkins-hbase4:41949] zookeeper.ReadOnlyZKClient(139): Connect 0x34bb383e to 127.0.0.1:49784 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 09:54:02,714 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 09:54:02,714 DEBUG [RS:0;jenkins-hbase4:40095] zookeeper.ReadOnlyZKClient(139): Connect 0x4d2905d0 to 127.0.0.1:49784 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 09:54:02,716 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 09:54:02,721 DEBUG [RS:1;jenkins-hbase4:34799] zookeeper.ReadOnlyZKClient(139): Connect 0x1f9b5be4 to 127.0.0.1:49784 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 09:54:02,726 DEBUG [RS:0;jenkins-hbase4:40095] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43eee205, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 09:54:02,726 DEBUG [RS:2;jenkins-hbase4:41949] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a7cc0d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 09:54:02,726 DEBUG [RS:0;jenkins-hbase4:40095] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47d16ea0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 09:54:02,727 DEBUG [RS:2;jenkins-hbase4:41949] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e43534e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 09:54:02,732 DEBUG [RS:1;jenkins-hbase4:34799] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@115ca667, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 09:54:02,732 DEBUG [RS:1;jenkins-hbase4:34799] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15633094, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 09:54:02,750 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34799 2023-02-11 09:54:02,753 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:41949 2023-02-11 09:54:02,753 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40095 2023-02-11 09:54:02,753 INFO [RS:1;jenkins-hbase4:34799] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 09:54:02,754 INFO [RS:1;jenkins-hbase4:34799] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 09:54:02,754 INFO [RS:0;jenkins-hbase4:40095] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 09:54:02,754 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 09:54:02,754 INFO [RS:2;jenkins-hbase4:41949] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 09:54:02,754 INFO [RS:0;jenkins-hbase4:40095] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 09:54:02,754 INFO [RS:2;jenkins-hbase4:41949] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 09:54:02,754 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 09:54:02,754 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 09:54:02,757 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43041,1676109240552 with isa=jenkins-hbase4.apache.org/172.31.14.131:40095, startcode=1676109241701 2023-02-11 09:54:02,757 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43041,1676109240552 with isa=jenkins-hbase4.apache.org/172.31.14.131:41949, startcode=1676109241776 2023-02-11 09:54:02,757 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43041,1676109240552 with isa=jenkins-hbase4.apache.org/172.31.14.131:34799, startcode=1676109241745 2023-02-11 09:54:02,774 DEBUG [RS:1;jenkins-hbase4:34799] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 09:54:02,774 DEBUG [RS:0;jenkins-hbase4:40095] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 09:54:02,774 DEBUG [RS:2;jenkins-hbase4:41949] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 09:54:02,830 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38781, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 09:54:02,830 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36867, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 09:54:02,830 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38439, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 09:54:02,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:02,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43041] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 09:54:02,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43041] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 09:54:02,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 09:54:02,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 09:54:02,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 09:54:02,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 09:54:02,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 09:54:02,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-11 09:54:02,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:02,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 09:54:02,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:02,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109272842 2023-02-11 09:54:02,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-11 09:54:02,851 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:02,852 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-11 09:54:02,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-11 09:54:02,858 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 09:54:02,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-11 09:54:02,867 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-11 09:54:02,867 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-11 09:54:02,867 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-11 09:54:02,868 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:02,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-11 09:54:02,872 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-11 09:54:02,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-11 09:54:02,874 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 09:54:02,874 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 09:54:02,874 WARN [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 09:54:02,874 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 09:54:02,874 WARN [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 09:54:02,875 WARN [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 09:54:02,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-11 09:54:02,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-11 09:54:02,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676109242879,5,FailOnTimeoutGroup] 2023-02-11 09:54:02,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676109242879,5,FailOnTimeoutGroup] 2023-02-11 09:54:02,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:02,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-11 09:54:02,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:02,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:02,897 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:02,898 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:02,898 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16 2023-02-11 09:54:02,919 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:02,921 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 09:54:02,924 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/info 2023-02-11 09:54:02,925 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 09:54:02,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:02,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 09:54:02,929 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/rep_barrier 2023-02-11 09:54:02,929 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 09:54:02,930 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:02,930 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 09:54:02,932 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/table 2023-02-11 09:54:02,933 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 09:54:02,934 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:02,935 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740 2023-02-11 09:54:02,937 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740 2023-02-11 09:54:02,940 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 09:54:02,943 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 09:54:02,947 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:02,949 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60264475, jitterRate=-0.10198934376239777}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 09:54:02,949 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676109242919Initializing all the Stores at 1676109242921 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676109242921Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109242921Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676109242921Cleaning up temporary data from old regions at 1676109242938 (+17 ms)Cleaning up detritus from prior splits at 1676109242939 (+1 ms)Region opened successfully at 1676109242949 (+10 ms) 2023-02-11 09:54:02,949 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 09:54:02,949 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 09:54:02,949 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 09:54:02,949 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 09:54:02,949 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 09:54:02,950 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 09:54:02,951 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676109242949Disabling compacts and flushes for region at 1676109242949Disabling writes for close at 1676109242949Writing region close event to WAL at 1676109242950 (+1 ms)Closed at 1676109242950 2023-02-11 09:54:02,956 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:02,956 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-11 09:54:02,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-11 09:54:02,976 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43041,1676109240552 with isa=jenkins-hbase4.apache.org/172.31.14.131:34799, startcode=1676109241745 2023-02-11 09:54:02,976 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43041,1676109240552 with isa=jenkins-hbase4.apache.org/172.31.14.131:40095, startcode=1676109241701 2023-02-11 09:54:02,976 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43041,1676109240552 with isa=jenkins-hbase4.apache.org/172.31.14.131:41949, startcode=1676109241776 2023-02-11 09:54:02,981 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-11 09:54:02,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:02,983 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43041] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:02,984 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43041] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:02,984 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 09:54:02,990 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16 2023-02-11 09:54:02,990 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16 2023-02-11 09:54:02,990 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16 2023-02-11 09:54:02,990 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41137 2023-02-11 09:54:02,990 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41137 2023-02-11 09:54:02,990 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 09:54:02,990 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41137 2023-02-11 09:54:02,990 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 09:54:02,990 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 09:54:02,999 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 09:54:03,000 DEBUG [RS:0;jenkins-hbase4:40095] zookeeper.ZKUtil(162): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:03,000 DEBUG [RS:2;jenkins-hbase4:41949] zookeeper.ZKUtil(162): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:03,000 WARN [RS:0;jenkins-hbase4:40095] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 09:54:03,001 WARN [RS:2;jenkins-hbase4:41949] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 09:54:03,001 DEBUG [RS:1;jenkins-hbase4:34799] zookeeper.ZKUtil(162): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,001 INFO [RS:2;jenkins-hbase4:41949] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 09:54:03,001 INFO [RS:0;jenkins-hbase4:40095] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 09:54:03,001 WARN [RS:1;jenkins-hbase4:34799] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 09:54:03,002 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:03,002 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40095,1676109241701] 2023-02-11 09:54:03,002 INFO [RS:1;jenkins-hbase4:34799] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 09:54:03,002 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:03,002 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41949,1676109241776] 2023-02-11 09:54:03,002 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,002 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34799,1676109241745] 2023-02-11 09:54:03,014 DEBUG [RS:1;jenkins-hbase4:34799] zookeeper.ZKUtil(162): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:03,014 DEBUG [RS:2;jenkins-hbase4:41949] zookeeper.ZKUtil(162): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:03,014 DEBUG [RS:0;jenkins-hbase4:40095] zookeeper.ZKUtil(162): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:03,014 DEBUG [RS:1;jenkins-hbase4:34799] zookeeper.ZKUtil(162): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,014 DEBUG [RS:2;jenkins-hbase4:41949] zookeeper.ZKUtil(162): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,015 DEBUG [RS:0;jenkins-hbase4:40095] zookeeper.ZKUtil(162): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,015 DEBUG [RS:1;jenkins-hbase4:34799] zookeeper.ZKUtil(162): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:03,015 DEBUG [RS:2;jenkins-hbase4:41949] zookeeper.ZKUtil(162): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:03,015 DEBUG [RS:0;jenkins-hbase4:40095] zookeeper.ZKUtil(162): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:03,025 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 09:54:03,025 DEBUG [RS:0;jenkins-hbase4:40095] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 09:54:03,025 DEBUG [RS:2;jenkins-hbase4:41949] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 09:54:03,034 INFO [RS:1;jenkins-hbase4:34799] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 09:54:03,034 INFO [RS:2;jenkins-hbase4:41949] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 09:54:03,034 INFO [RS:0;jenkins-hbase4:40095] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 09:54:03,053 INFO [RS:2;jenkins-hbase4:41949] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 09:54:03,053 INFO [RS:0;jenkins-hbase4:40095] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 09:54:03,053 INFO [RS:1;jenkins-hbase4:34799] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 09:54:03,056 INFO [RS:0;jenkins-hbase4:40095] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 09:54:03,056 INFO [RS:2;jenkins-hbase4:41949] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 09:54:03,057 INFO [RS:0;jenkins-hbase4:40095] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,056 INFO [RS:1;jenkins-hbase4:34799] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 09:54:03,057 INFO [RS:2;jenkins-hbase4:41949] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,058 INFO [RS:1;jenkins-hbase4:34799] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,058 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 09:54:03,059 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 09:54:03,059 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 09:54:03,067 INFO [RS:1;jenkins-hbase4:34799] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,067 INFO [RS:2;jenkins-hbase4:41949] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,067 INFO [RS:0;jenkins-hbase4:40095] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,067 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,067 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 09:54:03,068 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,068 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 09:54:03,069 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 09:54:03,069 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:2;jenkins-hbase4:41949] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:1;jenkins-hbase4:34799] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,069 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,072 DEBUG [RS:0;jenkins-hbase4:40095] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 09:54:03,077 INFO [RS:1;jenkins-hbase4:34799] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,077 INFO [RS:1;jenkins-hbase4:34799] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,077 INFO [RS:1;jenkins-hbase4:34799] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,077 INFO [RS:2;jenkins-hbase4:41949] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,077 INFO [RS:2;jenkins-hbase4:41949] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,078 INFO [RS:2;jenkins-hbase4:41949] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,079 INFO [RS:0;jenkins-hbase4:40095] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,079 INFO [RS:0;jenkins-hbase4:40095] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,079 INFO [RS:0;jenkins-hbase4:40095] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,097 INFO [RS:0;jenkins-hbase4:40095] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 09:54:03,097 INFO [RS:1;jenkins-hbase4:34799] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 09:54:03,097 INFO [RS:2;jenkins-hbase4:41949] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 09:54:03,100 INFO [RS:0;jenkins-hbase4:40095] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40095,1676109241701-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,100 INFO [RS:2;jenkins-hbase4:41949] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41949,1676109241776-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,100 INFO [RS:1;jenkins-hbase4:34799] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34799,1676109241745-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,123 INFO [RS:0;jenkins-hbase4:40095] regionserver.Replication(203): jenkins-hbase4.apache.org,40095,1676109241701 started 2023-02-11 09:54:03,124 INFO [RS:2;jenkins-hbase4:41949] regionserver.Replication(203): jenkins-hbase4.apache.org,41949,1676109241776 started 2023-02-11 09:54:03,124 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,40095,1676109241701, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40095, sessionid=0x10042cd3f510001 2023-02-11 09:54:03,124 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,41949,1676109241776, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41949, sessionid=0x10042cd3f510003 2023-02-11 09:54:03,124 INFO [RS:1;jenkins-hbase4:34799] regionserver.Replication(203): jenkins-hbase4.apache.org,34799,1676109241745 started 2023-02-11 09:54:03,124 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 09:54:03,124 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 09:54:03,124 DEBUG [RS:0;jenkins-hbase4:40095] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:03,124 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34799,1676109241745, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34799, sessionid=0x10042cd3f510002 2023-02-11 09:54:03,125 DEBUG [RS:0;jenkins-hbase4:40095] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40095,1676109241701' 2023-02-11 09:54:03,125 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 09:54:03,124 DEBUG [RS:2;jenkins-hbase4:41949] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:03,125 DEBUG [RS:1;jenkins-hbase4:34799] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,126 DEBUG [RS:1;jenkins-hbase4:34799] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34799,1676109241745' 2023-02-11 09:54:03,126 DEBUG [RS:1;jenkins-hbase4:34799] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:03,125 DEBUG [RS:0;jenkins-hbase4:40095] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:03,125 DEBUG [RS:2;jenkins-hbase4:41949] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41949,1676109241776' 2023-02-11 09:54:03,126 DEBUG [RS:2;jenkins-hbase4:41949] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:03,126 DEBUG [RS:1;jenkins-hbase4:34799] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:03,126 DEBUG [RS:0;jenkins-hbase4:40095] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:03,126 DEBUG [RS:2;jenkins-hbase4:41949] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:03,127 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 09:54:03,127 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 09:54:03,127 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 09:54:03,127 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 09:54:03,127 DEBUG [RS:1;jenkins-hbase4:34799] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,127 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 09:54:03,127 DEBUG [RS:1;jenkins-hbase4:34799] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34799,1676109241745' 2023-02-11 09:54:03,127 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 09:54:03,129 DEBUG [RS:0;jenkins-hbase4:40095] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:03,129 DEBUG [RS:0;jenkins-hbase4:40095] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40095,1676109241701' 2023-02-11 09:54:03,129 DEBUG [RS:0;jenkins-hbase4:40095] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 09:54:03,129 DEBUG [RS:1;jenkins-hbase4:34799] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 09:54:03,128 DEBUG [RS:2;jenkins-hbase4:41949] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:03,129 DEBUG [RS:2;jenkins-hbase4:41949] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41949,1676109241776' 2023-02-11 09:54:03,129 DEBUG [RS:2;jenkins-hbase4:41949] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 09:54:03,129 DEBUG [RS:0;jenkins-hbase4:40095] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 09:54:03,129 DEBUG [RS:1;jenkins-hbase4:34799] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 09:54:03,130 DEBUG [RS:2;jenkins-hbase4:41949] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 09:54:03,130 DEBUG [RS:0;jenkins-hbase4:40095] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 09:54:03,130 DEBUG [RS:1;jenkins-hbase4:34799] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 09:54:03,130 INFO [RS:0;jenkins-hbase4:40095] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 09:54:03,130 INFO [RS:1;jenkins-hbase4:34799] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 09:54:03,130 INFO [RS:1;jenkins-hbase4:34799] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 09:54:03,130 INFO [RS:0;jenkins-hbase4:40095] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 09:54:03,130 DEBUG [RS:2;jenkins-hbase4:41949] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 09:54:03,131 INFO [RS:2;jenkins-hbase4:41949] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 09:54:03,131 INFO [RS:2;jenkins-hbase4:41949] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 09:54:03,137 DEBUG [jenkins-hbase4:43041] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-11 09:54:03,140 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:03,147 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:03,147 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:03,147 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:03,147 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:03,149 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34799,1676109241745, state=OPENING 2023-02-11 09:54:03,156 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-11 09:54:03,161 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:03,162 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 09:54:03,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:03,242 INFO [RS:2;jenkins-hbase4:41949] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41949%2C1676109241776, suffix=, logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,41949,1676109241776, archiveDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs, maxLogs=32 2023-02-11 09:54:03,242 INFO [RS:1;jenkins-hbase4:34799] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34799%2C1676109241745, suffix=, logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,34799,1676109241745, archiveDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs, maxLogs=32 2023-02-11 09:54:03,242 INFO [RS:0;jenkins-hbase4:40095] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40095%2C1676109241701, suffix=, logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,40095,1676109241701, archiveDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs, maxLogs=32 2023-02-11 09:54:03,264 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK] 2023-02-11 09:54:03,265 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK] 2023-02-11 09:54:03,265 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK] 2023-02-11 09:54:03,274 INFO [RS:2;jenkins-hbase4:41949] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,41949,1676109241776/jenkins-hbase4.apache.org%2C41949%2C1676109241776.1676109243245 2023-02-11 09:54:03,274 DEBUG [RS:2;jenkins-hbase4:41949] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK], DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK], DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK]] 2023-02-11 09:54:03,285 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK] 2023-02-11 09:54:03,286 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK] 2023-02-11 09:54:03,286 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK] 2023-02-11 09:54:03,286 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK] 2023-02-11 09:54:03,286 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK] 2023-02-11 09:54:03,287 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK] 2023-02-11 09:54:03,297 INFO [RS:0;jenkins-hbase4:40095] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,40095,1676109241701/jenkins-hbase4.apache.org%2C40095%2C1676109241701.1676109243246 2023-02-11 09:54:03,300 DEBUG [RS:0;jenkins-hbase4:40095] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK], DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK], DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK]] 2023-02-11 09:54:03,301 INFO [RS:1;jenkins-hbase4:34799] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,34799,1676109241745/jenkins-hbase4.apache.org%2C34799%2C1676109241745.1676109243246 2023-02-11 09:54:03,301 DEBUG [RS:1;jenkins-hbase4:34799] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK], DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK], DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK]] 2023-02-11 09:54:03,349 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:03,351 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 09:54:03,355 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60200, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 09:54:03,367 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-11 09:54:03,367 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 09:54:03,371 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34799%2C1676109241745.meta, suffix=.meta, logDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,34799,1676109241745, archiveDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs, maxLogs=32 2023-02-11 09:54:03,392 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK] 2023-02-11 09:54:03,392 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK] 2023-02-11 09:54:03,394 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK] 2023-02-11 09:54:03,399 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/WALs/jenkins-hbase4.apache.org,34799,1676109241745/jenkins-hbase4.apache.org%2C34799%2C1676109241745.meta.1676109243372.meta 2023-02-11 09:54:03,404 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43349,DS-61d366ad-15e1-4269-bcaa-811227d6beff,DISK], DatanodeInfoWithStorage[127.0.0.1:36925,DS-1bb7723f-a65c-489e-b4e2-ba9b7fefb1f0,DISK], DatanodeInfoWithStorage[127.0.0.1:37777,DS-3e692238-0b4a-4eaa-997e-4c3ec947668d,DISK]] 2023-02-11 09:54:03,404 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:03,408 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:03,410 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-11 09:54:03,435 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-11 09:54:03,437 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-11 09:54:03,442 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-11 09:54:03,443 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:03,443 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-11 09:54:03,443 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-11 09:54:03,446 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 09:54:03,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/info 2023-02-11 09:54:03,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/info 2023-02-11 09:54:03,448 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 09:54:03,449 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:03,450 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 09:54:03,451 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/rep_barrier 2023-02-11 09:54:03,451 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/rep_barrier 2023-02-11 09:54:03,452 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 09:54:03,452 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:03,453 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 09:54:03,454 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/table 2023-02-11 09:54:03,454 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/table 2023-02-11 09:54:03,455 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 09:54:03,456 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:03,458 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740 2023-02-11 09:54:03,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740 2023-02-11 09:54:03,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 09:54:03,466 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 09:54:03,468 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67127420, jitterRate=2.765059471130371E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 09:54:03,468 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676109243443Writing region info on filesystem at 1676109243444 (+1 ms)Initializing all the Stores at 1676109243445 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676109243446 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109243446Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676109243446Cleaning up temporary data from old regions at 1676109243462 (+16 ms)Cleaning up detritus from prior splits at 1676109243463 (+1 ms)Running coprocessor post-open hooks at 1676109243468 (+5 ms)Region opened successfully at 1676109243468 2023-02-11 09:54:03,478 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676109243341 2023-02-11 09:54:03,497 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-11 09:54:03,498 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-11 09:54:03,499 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34799,1676109241745, state=OPEN 2023-02-11 09:54:03,502 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-11 09:54:03,502 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 09:54:03,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-11 09:54:03,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34799,1676109241745 in 337 msec 2023-02-11 09:54:03,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-11 09:54:03,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 543 msec 2023-02-11 09:54:03,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 776 msec 2023-02-11 09:54:03,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676109241818, completionTime=-1 2023-02-11 09:54:03,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-11 09:54:03,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:03,585 DEBUG [hconnection-0x6a75bb5c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 09:54:03,588 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38428, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 09:54:03,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:03,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109303606 2023-02-11 09:54:03,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109363606 2023-02-11 09:54:03,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 86 msec 2023-02-11 09:54:03,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43041,1676109240552-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43041,1676109240552-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43041,1676109240552-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43041, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-11 09:54:03,644 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-11 09:54:03,653 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-11 09:54:03,654 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 09:54:03,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:03,667 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:54:03,670 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:54:03,694 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:03,696 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06 empty. 2023-02-11 09:54:03,697 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:03,697 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-11 09:54:03,764 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:03,770 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8df39b0c7ffdf9ffd52d48b4d3daaa06, NAME => 'hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:54:03,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:03,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 8df39b0c7ffdf9ffd52d48b4d3daaa06, disabling compactions & flushes 2023-02-11 09:54:03,791 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:54:03,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:54:03,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. after waiting 0 ms 2023-02-11 09:54:03,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:54:03,791 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:54:03,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 8df39b0c7ffdf9ffd52d48b4d3daaa06: Waiting for close lock at 1676109243791Disabling compacts and flushes for region at 1676109243791Disabling writes for close at 1676109243791Writing region close event to WAL at 1676109243791Closed at 1676109243791 2023-02-11 09:54:03,795 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:54:03,809 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676109243798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109243798"}]},"ts":"1676109243798"} 2023-02-11 09:54:03,836 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:54:03,837 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:54:03,841 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109243838"}]},"ts":"1676109243838"} 2023-02-11 09:54:03,845 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-11 09:54:03,850 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:03,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:03,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:03,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:03,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:03,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8df39b0c7ffdf9ffd52d48b4d3daaa06, ASSIGN}] 2023-02-11 09:54:03,857 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8df39b0c7ffdf9ffd52d48b4d3daaa06, ASSIGN 2023-02-11 09:54:03,859 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=8df39b0c7ffdf9ffd52d48b4d3daaa06, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=false 2023-02-11 09:54:04,011 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:04,012 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:04,013 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676109244012"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109244012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109244012"}]},"ts":"1676109244012"} 2023-02-11 09:54:04,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 8df39b0c7ffdf9ffd52d48b4d3daaa06, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:04,176 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:54:04,177 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8df39b0c7ffdf9ffd52d48b4d3daaa06, NAME => 'hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:04,178 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:04,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:04,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:04,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:04,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:04,181 INFO [StoreOpener-8df39b0c7ffdf9ffd52d48b4d3daaa06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:04,183 DEBUG [StoreOpener-8df39b0c7ffdf9ffd52d48b4d3daaa06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/info 2023-02-11 09:54:04,183 DEBUG [StoreOpener-8df39b0c7ffdf9ffd52d48b4d3daaa06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/info 2023-02-11 09:54:04,184 INFO [StoreOpener-8df39b0c7ffdf9ffd52d48b4d3daaa06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8df39b0c7ffdf9ffd52d48b4d3daaa06 columnFamilyName info 2023-02-11 09:54:04,185 INFO [StoreOpener-8df39b0c7ffdf9ffd52d48b4d3daaa06-1] regionserver.HStore(310): Store=8df39b0c7ffdf9ffd52d48b4d3daaa06/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:04,186 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:04,187 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:04,191 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:54:04,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:04,195 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8df39b0c7ffdf9ffd52d48b4d3daaa06; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68909416, jitterRate=0.026830315589904785}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 09:54:04,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8df39b0c7ffdf9ffd52d48b4d3daaa06: Running coprocessor pre-open hook at 1676109244179Writing region info on filesystem at 1676109244179Initializing all the Stores at 1676109244181 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676109244181Cleaning up temporary data from old regions at 1676109244188 (+7 ms)Cleaning up detritus from prior splits at 1676109244188Running coprocessor post-open hooks at 1676109244195 (+7 ms)Region opened successfully at 1676109244195 2023-02-11 09:54:04,197 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06., pid=6, masterSystemTime=1676109244170 2023-02-11 09:54:04,200 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:54:04,200 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:54:04,202 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:04,202 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676109244201"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109244201"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109244201"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109244201"}]},"ts":"1676109244201"} 2023-02-11 09:54:04,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-11 09:54:04,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 8df39b0c7ffdf9ffd52d48b4d3daaa06, server=jenkins-hbase4.apache.org,34799,1676109241745 in 190 msec 2023-02-11 09:54:04,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-11 09:54:04,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=8df39b0c7ffdf9ffd52d48b4d3daaa06, ASSIGN in 356 msec 2023-02-11 09:54:04,215 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:54:04,215 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109244215"}]},"ts":"1676109244215"} 2023-02-11 09:54:04,217 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-11 09:54:04,221 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:54:04,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 565 msec 2023-02-11 09:54:04,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-11 09:54:04,270 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-11 09:54:04,270 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:04,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:04,317 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 09:54:04,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 27 msec 2023-02-11 09:54:04,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:04,349 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 09:54:04,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-02-11 09:54:04,361 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-11 09:54:04,364 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-11 09:54:04,364 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.533sec 2023-02-11 09:54:04,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-11 09:54:04,369 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-11 09:54:04,369 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-11 09:54:04,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43041,1676109240552-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-11 09:54:04,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43041,1676109240552-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-11 09:54:04,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-11 09:54:04,401 DEBUG [Listener at localhost/34597] zookeeper.ReadOnlyZKClient(139): Connect 0x37e887de to 127.0.0.1:49784 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 09:54:04,405 DEBUG [Listener at localhost/34597] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c8d5aec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 09:54:04,419 DEBUG [hconnection-0x7e2f5f25-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 09:54:04,430 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38442, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 09:54:04,439 INFO [Listener at localhost/34597] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:54:04,471 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=359, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=169, AvailableMemoryMB=3103 2023-02-11 09:54:04,475 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:04,483 DEBUG [Listener at localhost/34597] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 09:54:04,486 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52560, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 09:54:04,499 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:54:04,499 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:54:04,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:54:04,505 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:04,507 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:54:04,509 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:54:04,512 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,512 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218 empty. 2023-02-11 09:54:04,514 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,515 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 09:54:04,535 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:04,537 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => d216b600c0cb9961b975974a1b3ad218, NAME => 'testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:54:04,554 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:04,554 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing d216b600c0cb9961b975974a1b3ad218, disabling compactions & flushes 2023-02-11 09:54:04,554 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:04,554 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:04,555 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. after waiting 0 ms 2023-02-11 09:54:04,555 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:04,555 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:04,555 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for d216b600c0cb9961b975974a1b3ad218: Waiting for close lock at 1676109244554Disabling compacts and flushes for region at 1676109244554Disabling writes for close at 1676109244555 (+1 ms)Writing region close event to WAL at 1676109244555Closed at 1676109244555 2023-02-11 09:54:04,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:54:04,560 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676109244560"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109244560"}]},"ts":"1676109244560"} 2023-02-11 09:54:04,562 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:54:04,564 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:54:04,564 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109244564"}]},"ts":"1676109244564"} 2023-02-11 09:54:04,566 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-11 09:54:04,570 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:04,571 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:04,571 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:04,571 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:04,571 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:04,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d216b600c0cb9961b975974a1b3ad218, ASSIGN}] 2023-02-11 09:54:04,574 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d216b600c0cb9961b975974a1b3ad218, ASSIGN 2023-02-11 09:54:04,575 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d216b600c0cb9961b975974a1b3ad218, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41949,1676109241776; forceNewPlan=false, retain=false 2023-02-11 09:54:04,725 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:04,726 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d216b600c0cb9961b975974a1b3ad218, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:04,727 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676109244726"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109244726"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109244726"}]},"ts":"1676109244726"} 2023-02-11 09:54:04,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure d216b600c0cb9961b975974a1b3ad218, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:54:04,884 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:04,884 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 09:54:04,888 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52812, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 09:54:04,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:04,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d216b600c0cb9961b975974a1b3ad218, NAME => 'testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:04,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:04,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:04,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,900 INFO [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,902 DEBUG [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1 2023-02-11 09:54:04,902 DEBUG [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1 2023-02-11 09:54:04,902 INFO [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d216b600c0cb9961b975974a1b3ad218 columnFamilyName cf1 2023-02-11 09:54:04,903 INFO [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] regionserver.HStore(310): Store=d216b600c0cb9961b975974a1b3ad218/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:04,903 INFO [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,905 DEBUG [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2 2023-02-11 09:54:04,905 DEBUG [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2 2023-02-11 09:54:04,906 INFO [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d216b600c0cb9961b975974a1b3ad218 columnFamilyName cf2 2023-02-11 09:54:04,906 INFO [StoreOpener-d216b600c0cb9961b975974a1b3ad218-1] regionserver.HStore(310): Store=d216b600c0cb9961b975974a1b3ad218/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:04,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:04,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:04,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:04,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d216b600c0cb9961b975974a1b3ad218; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59782325, jitterRate=-0.10917393863201141}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:04,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d216b600c0cb9961b975974a1b3ad218: Running coprocessor pre-open hook at 1676109244895Writing region info on filesystem at 1676109244895Initializing all the Stores at 1676109244896 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109244897 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109244899 (+2 ms)Cleaning up temporary data from old regions at 1676109244910 (+11 ms)Cleaning up detritus from prior splits at 1676109244911 (+1 ms)Running coprocessor post-open hooks at 1676109244918 (+7 ms)Region opened successfully at 1676109244918 2023-02-11 09:54:04,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218., pid=11, masterSystemTime=1676109244884 2023-02-11 09:54:04,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:04,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:04,925 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d216b600c0cb9961b975974a1b3ad218, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:04,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676109244925"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109244925"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109244925"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109244925"}]},"ts":"1676109244925"} 2023-02-11 09:54:04,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-11 09:54:04,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure d216b600c0cb9961b975974a1b3ad218, server=jenkins-hbase4.apache.org,41949,1676109241776 in 200 msec 2023-02-11 09:54:04,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-11 09:54:04,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d216b600c0cb9961b975974a1b3ad218, ASSIGN in 361 msec 2023-02-11 09:54:04,936 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:54:04,936 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109244936"}]},"ts":"1676109244936"} 2023-02-11 09:54:04,939 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-11 09:54:04,942 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:54:04,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 438 msec 2023-02-11 09:54:05,013 DEBUG [Listener at localhost/34597] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 09:54:05,019 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52822, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 09:54:05,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:05,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,051 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,051 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 09:54:05,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 09:54:05,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,054 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,054 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,054 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,054 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,055 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,055 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:05,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,059 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,059 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,059 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,059 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,062 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:05,062 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:05,062 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:05,062 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. started... 2023-02-11 09:54:05,063 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d216b600c0cb9961b975974a1b3ad218 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:54:05,066 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,067 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,067 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,067 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,067 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,067 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,138 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/b527bc84bea146f79704b40b48b99dc6 2023-02-11 09:54:05,198 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/fb42258a2612413fa71c12551940164a 2023-02-11 09:54:05,209 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/b527bc84bea146f79704b40b48b99dc6 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6 2023-02-11 09:54:05,219 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:05,222 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/fb42258a2612413fa71c12551940164a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a 2023-02-11 09:54:05,231 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:05,233 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 170ms, sequenceid=6, compaction requested=false 2023-02-11 09:54:05,234 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,236 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d216b600c0cb9961b975974a1b3ad218: 2023-02-11 09:54:05,236 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:05,236 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:05,237 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:05,237 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,237 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,237 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,240 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,240 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,240 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,240 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 09:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9a7cc59[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:05,243 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:05,244 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,250 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,250 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,250 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:05,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,250 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:05,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:05,254 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,256 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,256 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,256 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:05,262 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 09:54:05,262 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:05,262 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:05,262 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,262 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:05,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,268 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,268 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,269 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,271 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,272 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,272 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,284 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,284 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,284 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,284 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,285 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,284 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,285 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,285 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,285 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,285 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,285 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,285 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,287 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,287 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,287 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,287 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,288 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:05,288 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:05,388 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:05,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:05,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:05,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:05,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,420 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,421 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,421 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 09:54:05,421 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 09:54:05,422 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,422 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,423 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,423 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,423 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,423 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,424 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,424 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,425 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,426 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,426 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,427 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,427 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,427 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,427 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,427 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,428 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,428 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,428 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,428 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,427 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,428 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:05,428 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:05,428 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:05,428 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. started... 2023-02-11 09:54:05,429 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d216b600c0cb9961b975974a1b3ad218 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:05,430 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,430 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,431 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,431 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,431 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,431 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,458 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/f8b40a2d8e9045549cad4b35fe58856b 2023-02-11 09:54:05,486 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9 2023-02-11 09:54:05,495 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/f8b40a2d8e9045549cad4b35fe58856b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b 2023-02-11 09:54:05,502 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:05,504 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9 2023-02-11 09:54:05,513 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:05,515 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 86ms, sequenceid=19, compaction requested=false 2023-02-11 09:54:05,515 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d216b600c0cb9961b975974a1b3ad218: 2023-02-11 09:54:05,515 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,519 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,519 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,520 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,520 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 09:54:05,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6561e879[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:05,523 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:05,523 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,526 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,526 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:05,526 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:05,529 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,529 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,529 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,529 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,530 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,530 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,531 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:54:05,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:05,535 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 09:54:05,535 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:05,535 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:05,535 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,535 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:05,537 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,537 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,537 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,537 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,538 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,539 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,539 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,542 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,543 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,544 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,553 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,553 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,553 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,554 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,554 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,554 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,553 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,554 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,554 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,554 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,554 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:05,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,554 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,556 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:05,556 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:05,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,656 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:05,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:05,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:05,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:05,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,691 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,691 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,692 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 09:54:05,692 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 09:54:05,692 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,692 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,694 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,694 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,694 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,698 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,698 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,698 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,698 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,699 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,699 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,699 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:05,699 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:05,699 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:05,700 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. started... 2023-02-11 09:54:05,700 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d216b600c0cb9961b975974a1b3ad218 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:05,701 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,701 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,701 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,702 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,703 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,703 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,723 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/200a5de99e0a419186b8d1c81286386b 2023-02-11 09:54:05,752 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/029016d368a24b1097e79e17a9fb2819 2023-02-11 09:54:05,762 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/200a5de99e0a419186b8d1c81286386b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b 2023-02-11 09:54:05,770 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:05,771 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/029016d368a24b1097e79e17a9fb2819 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819 2023-02-11 09:54:05,778 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:05,780 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 80ms, sequenceid=32, compaction requested=true 2023-02-11 09:54:05,780 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d216b600c0cb9961b975974a1b3ad218: 2023-02-11 09:54:05,780 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:05,780 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:05,781 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:05,781 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,781 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,781 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,784 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,784 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,784 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,784 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 09:54:05,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1df2cbb8[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:05,787 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:05,787 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,788 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,788 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,788 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:05,788 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,789 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,789 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:05,789 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:05,789 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:05,789 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:05,790 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,791 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,791 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,791 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,791 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,791 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,791 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:05,792 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,792 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,792 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:54:05,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:05,797 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 09:54:05,797 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:05,797 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:05,797 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,797 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:05,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:05,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,807 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,808 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,809 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:05,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,812 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:05,812 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,813 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,820 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,820 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,821 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,821 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:05,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,821 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:05,821 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,821 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,821 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,821 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:05,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:05,822 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:05,922 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:05,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:05,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:05,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:05,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,964 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:05,964 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,965 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 09:54:05,965 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 09:54:05,965 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,966 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,967 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,967 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,967 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,967 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:05,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:05,968 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:05,968 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,970 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,970 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,971 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,971 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,971 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,972 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,972 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,972 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,972 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,972 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:05,972 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,972 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:05,973 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,974 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:05,974 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,974 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:05,974 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:05,974 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:05,975 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:05,975 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:05,975 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,975 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. started... 2023-02-11 09:54:05,975 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d216b600c0cb9961b975974a1b3ad218 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:05,975 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,977 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,978 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:05,978 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:05,998 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/ef1d1698733b4961b6b2b3742f70261a 2023-02-11 09:54:06,023 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/09b93cb8cc6f4d429130dc66ba8a189e 2023-02-11 09:54:06,032 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/ef1d1698733b4961b6b2b3742f70261a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a 2023-02-11 09:54:06,040 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:06,041 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/09b93cb8cc6f4d429130dc66ba8a189e as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e 2023-02-11 09:54:06,049 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:06,050 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 75ms, sequenceid=45, compaction requested=true 2023-02-11 09:54:06,050 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d216b600c0cb9961b975974a1b3ad218: 2023-02-11 09:54:06,050 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,051 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:06,051 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:06,051 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,052 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:06,052 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:06,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,056 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,056 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,056 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:06,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 09:54:06,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dff7691[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:06,058 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:06,058 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,060 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,060 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:06,060 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,060 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,060 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:06,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,060 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:06,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,061 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,061 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,061 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:06,063 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,063 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,063 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,064 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,064 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,065 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,065 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,065 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,065 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,065 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:54:06,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:06,070 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 09:54:06,070 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:06,070 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:06,070 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,070 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:06,072 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,072 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,072 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,072 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,072 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,072 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,073 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,074 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,074 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,076 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,077 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,077 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:06,081 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,082 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,082 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:06,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,082 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:06,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,082 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,083 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,083 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,083 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,083 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,083 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,083 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,083 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,182 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:06,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:06,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:06,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:06,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,208 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:06,208 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,209 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 09:54:06,209 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 09:54:06,210 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,210 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,212 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,212 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,212 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,212 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,213 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,213 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:06,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,215 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:06,215 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:06,215 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:06,215 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:06,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:06,218 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:06,218 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:06,218 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:06,218 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:06,218 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,220 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:06,220 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:06,220 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:06,220 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,220 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. started... 2023-02-11 09:54:06,220 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d216b600c0cb9961b975974a1b3ad218 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:06,221 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,221 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,222 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,222 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:06,222 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,222 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:06,237 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4 2023-02-11 09:54:06,267 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/ec260062bfe64e64bf59e377422e61bb 2023-02-11 09:54:06,276 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4 2023-02-11 09:54:06,283 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:06,284 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/ec260062bfe64e64bf59e377422e61bb as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/ec260062bfe64e64bf59e377422e61bb 2023-02-11 09:54:06,292 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/ec260062bfe64e64bf59e377422e61bb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:06,293 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 73ms, sequenceid=58, compaction requested=true 2023-02-11 09:54:06,293 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d216b600c0cb9961b975974a1b3ad218: 2023-02-11 09:54:06,293 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,293 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:06,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:06,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:06,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:06,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,296 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,297 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,297 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:06,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 09:54:06,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a0db473[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:06,299 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:06,299 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,300 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,300 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,300 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,301 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,301 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:06,301 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:06,304 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,304 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,304 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,305 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,305 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,305 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,305 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,305 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,306 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,306 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:54:06,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:06,311 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 09:54:06,311 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:06,311 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:06,311 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,311 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:06,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,314 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,314 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,315 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,316 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,317 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,317 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,323 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:06,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,323 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,324 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,324 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:06,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,324 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,324 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:06,324 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,325 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,325 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,325 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,325 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,325 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,325 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,425 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:06,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:06,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:06,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:06,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,450 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:06,450 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,451 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 09:54:06,451 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 09:54:06,451 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,451 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,452 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,452 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,453 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,453 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,453 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,453 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:06,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:06,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:06,455 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:06,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 09:54:06,455 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:06,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,455 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,455 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 09:54:06,456 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:06,456 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:06,456 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,455 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:06,456 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:06,457 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 09:54:06,456 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:06,457 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:06,457 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:06,457 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:06,457 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:06,457 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,457 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. started... 2023-02-11 09:54:06,458 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d216b600c0cb9961b975974a1b3ad218 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:06,459 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,459 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,459 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,459 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,459 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:06,459 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:06,474 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/9e6964fafed1465d875cbfa140fb8c77 2023-02-11 09:54:06,497 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/e0523ccf488f438d8ac8da6f93d6ca73 2023-02-11 09:54:06,505 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/9e6964fafed1465d875cbfa140fb8c77 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9e6964fafed1465d875cbfa140fb8c77 2023-02-11 09:54:06,512 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9e6964fafed1465d875cbfa140fb8c77, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:06,513 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/e0523ccf488f438d8ac8da6f93d6ca73 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/e0523ccf488f438d8ac8da6f93d6ca73 2023-02-11 09:54:06,520 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/e0523ccf488f438d8ac8da6f93d6ca73, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:06,521 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 63ms, sequenceid=71, compaction requested=true 2023-02-11 09:54:06,521 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d216b600c0cb9961b975974a1b3ad218: 2023-02-11 09:54:06,522 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,522 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:06,523 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:06,523 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,523 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 09:54:06,523 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 09:54:06,524 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,524 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,525 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,525 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 09:54:06,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4045b36b[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:06,527 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:06,527 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,530 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:06,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:06,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 09:54:06,530 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,532 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 09:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 09:54:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:54:06,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:06,538 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 09:54:06,538 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:06,538 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:06,538 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,538 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:06,540 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,540 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,540 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,540 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:06,541 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,541 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,543 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,543 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:06,544 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 09:54:06,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:06,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,548 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:06,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:06,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:06,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:06,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,550 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 09:54:06,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:06,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:06,650 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:06,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:06,673 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,680 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:06,681 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:06,681 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:06,681 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:06,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d216b600c0cb9961b975974a1b3ad218, UNASSIGN}] 2023-02-11 09:54:06,690 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d216b600c0cb9961b975974a1b3ad218, UNASSIGN 2023-02-11 09:54:06,691 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d216b600c0cb9961b975974a1b3ad218, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:06,691 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676109246690"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109246690"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109246690"}]},"ts":"1676109246690"} 2023-02-11 09:54:06,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure d216b600c0cb9961b975974a1b3ad218, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:54:06,853 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:06,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d216b600c0cb9961b975974a1b3ad218, disabling compactions & flushes 2023-02-11 09:54:06,853 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. after waiting 0 ms 2023-02-11 09:54:06,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,854 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing d216b600c0cb9961b975974a1b3ad218 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 09:54:06,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/9338520281ae498db7acfd4210009b4e 2023-02-11 09:54:06,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/84c382d07d654f428a78d3dd9839e7c2 2023-02-11 09:54:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf1/9338520281ae498db7acfd4210009b4e as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9338520281ae498db7acfd4210009b4e 2023-02-11 09:54:06,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9338520281ae498db7acfd4210009b4e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:54:06,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.tmp/cf2/84c382d07d654f428a78d3dd9839e7c2 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/84c382d07d654f428a78d3dd9839e7c2 2023-02-11 09:54:06,967 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/84c382d07d654f428a78d3dd9839e7c2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:54:06,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 115ms, sequenceid=82, compaction requested=true 2023-02-11 09:54:06,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 09:54:06,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:06,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. 2023-02-11 09:54:06,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d216b600c0cb9961b975974a1b3ad218: Waiting for close lock at 1676109246853Running coprocessor pre-close hooks at 1676109246853Disabling compacts and flushes for region at 1676109246853Disabling writes for close at 1676109246853Obtaining lock to block concurrent updates at 1676109246854 (+1 ms)Preparing flush snapshotting stores in d216b600c0cb9961b975974a1b3ad218 at 1676109246854Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676109246854Flushing stores of testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. at 1676109246855 (+1 ms)Flushing d216b600c0cb9961b975974a1b3ad218/cf1: creating writer at 1676109246855Flushing d216b600c0cb9961b975974a1b3ad218/cf1: appending metadata at 1676109246860 (+5 ms)Flushing d216b600c0cb9961b975974a1b3ad218/cf1: closing flushed file at 1676109246860Flushing d216b600c0cb9961b975974a1b3ad218/cf2: creating writer at 1676109246900 (+40 ms)Flushing d216b600c0cb9961b975974a1b3ad218/cf2: appending metadata at 1676109246904 (+4 ms)Flushing d216b600c0cb9961b975974a1b3ad218/cf2: closing flushed file at 1676109246905 (+1 ms)Flushing d216b600c0cb9961b975974a1b3ad218/cf1: reopening flushed file at 1676109246946 (+41 ms)Flushing d216b600c0cb9961b975974a1b3ad218/cf2: reopening flushed file at 1676109246958 (+12 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d216b600c0cb9961b975974a1b3ad218 in 115ms, sequenceid=82, compaction requested=true at 1676109246969 (+11 ms)Writing region close event to WAL at 1676109246992 (+23 ms)Running coprocessor post-close hooks at 1676109246998 (+6 ms)Closed at 1676109246998 2023-02-11 09:54:07,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,002 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d216b600c0cb9961b975974a1b3ad218, regionState=CLOSED 2023-02-11 09:54:07,003 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676109247002"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109247002"}]},"ts":"1676109247002"} 2023-02-11 09:54:07,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-11 09:54:07,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure d216b600c0cb9961b975974a1b3ad218, server=jenkins-hbase4.apache.org,41949,1676109241776 in 312 msec 2023-02-11 09:54:07,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-11 09:54:07,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d216b600c0cb9961b975974a1b3ad218, UNASSIGN in 320 msec 2023-02-11 09:54:07,032 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=d216b600c0cb9961b975974a1b3ad218, threads=14 2023-02-11 09:54:07,033 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,034 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,034 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,034 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/84c382d07d654f428a78d3dd9839e7c2 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,034 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/e0523ccf488f438d8ac8da6f93d6ca73 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,034 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/ec260062bfe64e64bf59e377422e61bb for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,035 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,036 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9338520281ae498db7acfd4210009b4e for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,036 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,036 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9e6964fafed1465d875cbfa140fb8c77 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,044 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,044 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,044 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,045 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,083 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/ec260062bfe64e64bf59e377422e61bb, top=true 2023-02-11 09:54:07,085 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/84c382d07d654f428a78d3dd9839e7c2, top=true 2023-02-11 09:54:07,087 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/e0523ccf488f438d8ac8da6f93d6ca73, top=true 2023-02-11 09:54:07,099 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4, top=true 2023-02-11 09:54:07,107 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9e6964fafed1465d875cbfa140fb8c77, top=true 2023-02-11 09:54:07,148 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9338520281ae498db7acfd4210009b4e, top=true 2023-02-11 09:54:07,158 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.splits/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-ec260062bfe64e64bf59e377422e61bb for child: 98077030a27e351168173ed0c22ff5a4, parent: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,158 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/ec260062bfe64e64bf59e377422e61bb for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,163 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.splits/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-84c382d07d654f428a78d3dd9839e7c2 for child: 98077030a27e351168173ed0c22ff5a4, parent: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,163 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/84c382d07d654f428a78d3dd9839e7c2 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,173 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.splits/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-e0523ccf488f438d8ac8da6f93d6ca73 for child: 98077030a27e351168173ed0c22ff5a4, parent: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,173 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/e0523ccf488f438d8ac8da6f93d6ca73 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,177 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.splits/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-fa7fe2dc831a48f2a9d27b690aacd0c4 for child: 98077030a27e351168173ed0c22ff5a4, parent: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,178 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,204 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.splits/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9338520281ae498db7acfd4210009b4e for child: 98077030a27e351168173ed0c22ff5a4, parent: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,204 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9338520281ae498db7acfd4210009b4e for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,204 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/.splits/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9e6964fafed1465d875cbfa140fb8c77 for child: 98077030a27e351168173ed0c22ff5a4, parent: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,205 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9e6964fafed1465d875cbfa140fb8c77 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,258 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,258 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,264 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,264 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6 for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,266 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,275 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,275 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,276 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b for region: d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:07,276 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region d216b600c0cb9961b975974a1b3ad218 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:54:07,335 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:54:07,338 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:54:07,340 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676109247340"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676109247340"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676109247340"}]},"ts":"1676109247340"} 2023-02-11 09:54:07,341 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109247340"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109247340"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109247340"}]},"ts":"1676109247340"} 2023-02-11 09:54:07,341 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109247340"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109247340"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109247340"}]},"ts":"1676109247340"} 2023-02-11 09:54:07,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b2c7ebbb9b8d1a8598f01c1b9431d43c, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=98077030a27e351168173ed0c22ff5a4, ASSIGN}] 2023-02-11 09:54:07,392 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=98077030a27e351168173ed0c22ff5a4, ASSIGN 2023-02-11 09:54:07,393 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=98077030a27e351168173ed0c22ff5a4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41949,1676109241776; forceNewPlan=false, retain=false 2023-02-11 09:54:07,394 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b2c7ebbb9b8d1a8598f01c1b9431d43c, ASSIGN 2023-02-11 09:54:07,395 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b2c7ebbb9b8d1a8598f01c1b9431d43c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41949,1676109241776; forceNewPlan=false, retain=false 2023-02-11 09:54:07,543 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 09:54:07,545 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=b2c7ebbb9b8d1a8598f01c1b9431d43c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:07,545 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109247545"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109247545"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109247545"}]},"ts":"1676109247545"} 2023-02-11 09:54:07,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure b2c7ebbb9b8d1a8598f01c1b9431d43c, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:54:07,552 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=98077030a27e351168173ed0c22ff5a4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:07,552 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109247552"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109247552"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109247552"}]},"ts":"1676109247552"} 2023-02-11 09:54:07,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 98077030a27e351168173ed0c22ff5a4, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:54:07,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:07,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b2c7ebbb9b8d1a8598f01c1b9431d43c, NAME => 'testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.', STARTKEY => '', ENDKEY => '26'} 2023-02-11 09:54:07,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:07,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:07,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,707 INFO [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,708 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1 2023-02-11 09:54:07,708 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1 2023-02-11 09:54:07,709 INFO [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2c7ebbb9b8d1a8598f01c1b9431d43c columnFamilyName cf1 2023-02-11 09:54:07,749 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b-bottom 2023-02-11 09:54:07,757 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6-bottom 2023-02-11 09:54:07,764 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a-bottom 2023-02-11 09:54:07,771 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b-bottom 2023-02-11 09:54:07,772 INFO [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(310): Store=b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:07,772 INFO [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,774 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2 2023-02-11 09:54:07,774 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2 2023-02-11 09:54:07,775 INFO [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2c7ebbb9b8d1a8598f01c1b9431d43c columnFamilyName cf2 2023-02-11 09:54:07,792 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819-bottom 2023-02-11 09:54:07,799 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e-bottom 2023-02-11 09:54:07,807 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9-bottom 2023-02-11 09:54:07,814 DEBUG [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a-bottom 2023-02-11 09:54:07,814 INFO [StoreOpener-b2c7ebbb9b8d1a8598f01c1b9431d43c-1] regionserver.HStore(310): Store=b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:07,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:07,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:07,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b2c7ebbb9b8d1a8598f01c1b9431d43c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63825396, jitterRate=-0.048927485942840576}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:07,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b2c7ebbb9b8d1a8598f01c1b9431d43c: Running coprocessor pre-open hook at 1676109247706Writing region info on filesystem at 1676109247706Initializing all the Stores at 1676109247707 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109247707Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109247707Cleaning up temporary data from old regions at 1676109247819 (+112 ms)Cleaning up detritus from prior splits at 1676109247820 (+1 ms)Running coprocessor post-open hooks at 1676109247823 (+3 ms)Region opened successfully at 1676109247823 2023-02-11 09:54:07,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c., pid=17, masterSystemTime=1676109247700 2023-02-11 09:54:07,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:07,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:07,829 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 09:54:07,829 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 09:54:07,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:07,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:07,832 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:07,832 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:07,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:07,832 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1912): b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2 is initiating minor compaction (all files) 2023-02-11 09:54:07,832 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1912): b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1 is initiating minor compaction (all files) 2023-02-11 09:54:07,833 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2 in testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:07,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 98077030a27e351168173ed0c22ff5a4, NAME => 'testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.', STARTKEY => '26', ENDKEY => ''} 2023-02-11 09:54:07,833 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1 in testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:07,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=b2c7ebbb9b8d1a8598f01c1b9431d43c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:07,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:07,833 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/.tmp, totalSize=19.9 K 2023-02-11 09:54:07,833 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/.tmp, totalSize=19.9 K 2023-02-11 09:54:07,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:07,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109247833"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109247833"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109247833"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109247833"}]},"ts":"1676109247833"} 2023-02-11 09:54:07,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:07,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:07,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:07,837 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109245019 2023-02-11 09:54:07,837 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109245019 2023-02-11 09:54:07,839 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109245396 2023-02-11 09:54:07,840 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109245396 2023-02-11 09:54:07,840 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109245659 2023-02-11 09:54:07,840 INFO [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:07,841 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109245659 2023-02-11 09:54:07,841 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109245925 2023-02-11 09:54:07,842 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109245925 2023-02-11 09:54:07,844 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1 2023-02-11 09:54:07,844 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1 2023-02-11 09:54:07,844 INFO [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 98077030a27e351168173ed0c22ff5a4 columnFamilyName cf1 2023-02-11 09:54:07,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-11 09:54:07,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure b2c7ebbb9b8d1a8598f01c1b9431d43c, server=jenkins-hbase4.apache.org,41949,1676109241776 in 291 msec 2023-02-11 09:54:07,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b2c7ebbb9b8d1a8598f01c1b9431d43c, ASSIGN in 454 msec 2023-02-11 09:54:07,873 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b-top 2023-02-11 09:54:07,874 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] throttle.PressureAwareThroughputController(145): b2c7ebbb9b8d1a8598f01c1b9431d43c#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:07,876 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] throttle.PressureAwareThroughputController(145): b2c7ebbb9b8d1a8598f01c1b9431d43c#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:07,881 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6-top 2023-02-11 09:54:07,888 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a-top 2023-02-11 09:54:07,894 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b-top 2023-02-11 09:54:07,921 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9338520281ae498db7acfd4210009b4e 2023-02-11 09:54:07,931 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9e6964fafed1465d875cbfa140fb8c77 2023-02-11 09:54:07,941 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-fa7fe2dc831a48f2a9d27b690aacd0c4 2023-02-11 09:54:07,973 INFO [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(310): Store=98077030a27e351168173ed0c22ff5a4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:07,973 INFO [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:07,980 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2 2023-02-11 09:54:07,980 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2 2023-02-11 09:54:07,980 INFO [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 98077030a27e351168173ed0c22ff5a4 columnFamilyName cf2 2023-02-11 09:54:07,987 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/.tmp/cf2/5e167d5bcf92453a916cd509671c2da4 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/5e167d5bcf92453a916cd509671c2da4 2023-02-11 09:54:07,994 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/.tmp/cf1/f51691fc097d49449df1043312b5b056 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f51691fc097d49449df1043312b5b056 2023-02-11 09:54:08,014 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1 of b2c7ebbb9b8d1a8598f01c1b9431d43c into f51691fc097d49449df1043312b5b056(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:08,015 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b2c7ebbb9b8d1a8598f01c1b9431d43c: 2023-02-11 09:54:08,015 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c., storeName=b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1, priority=12, startTime=1676109247827; duration=0sec 2023-02-11 09:54:08,015 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:08,016 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2 of b2c7ebbb9b8d1a8598f01c1b9431d43c into 5e167d5bcf92453a916cd509671c2da4(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:08,017 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b2c7ebbb9b8d1a8598f01c1b9431d43c: 2023-02-11 09:54:08,017 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c., storeName=b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2, priority=12, startTime=1676109247828; duration=0sec 2023-02-11 09:54:08,017 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:08,024 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819-top 2023-02-11 09:54:08,030 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e-top 2023-02-11 09:54:08,041 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9-top 2023-02-11 09:54:08,053 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a-top 2023-02-11 09:54:08,073 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-84c382d07d654f428a78d3dd9839e7c2 2023-02-11 09:54:08,079 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-e0523ccf488f438d8ac8da6f93d6ca73 2023-02-11 09:54:08,085 DEBUG [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-ec260062bfe64e64bf59e377422e61bb 2023-02-11 09:54:08,086 INFO [StoreOpener-98077030a27e351168173ed0c22ff5a4-1] regionserver.HStore(310): Store=98077030a27e351168173ed0c22ff5a4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:08,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:08,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:08,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:08,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:08,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 98077030a27e351168173ed0c22ff5a4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60152956, jitterRate=-0.10365110635757446}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:08,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 98077030a27e351168173ed0c22ff5a4: Running coprocessor pre-open hook at 1676109247837Writing region info on filesystem at 1676109247837Initializing all the Stores at 1676109247840 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109247840Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109247840Cleaning up temporary data from old regions at 1676109248090 (+250 ms)Cleaning up detritus from prior splits at 1676109248090Running coprocessor post-open hooks at 1676109248094 (+4 ms)Region opened successfully at 1676109248094 2023-02-11 09:54:08,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4., pid=18, masterSystemTime=1676109247700 2023-02-11 09:54:08,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:08,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:08,097 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:08,097 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:08,098 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,098 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,098 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1912): 98077030a27e351168173ed0c22ff5a4/cf1 is initiating minor compaction (all files) 2023-02-11 09:54:08,098 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1912): 98077030a27e351168173ed0c22ff5a4/cf2 is initiating minor compaction (all files) 2023-02-11 09:54:08,098 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 98077030a27e351168173ed0c22ff5a4/cf1 in testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,098 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 98077030a27e351168173ed0c22ff5a4/cf2 in testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,099 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-ec260062bfe64e64bf59e377422e61bb, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-e0523ccf488f438d8ac8da6f93d6ca73, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-84c382d07d654f428a78d3dd9839e7c2] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/.tmp, totalSize=34.9 K 2023-02-11 09:54:08,099 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-fa7fe2dc831a48f2a9d27b690aacd0c4, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9e6964fafed1465d875cbfa140fb8c77, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9338520281ae498db7acfd4210009b4e] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/.tmp, totalSize=34.9 K 2023-02-11 09:54:08,099 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109245019 2023-02-11 09:54:08,101 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109245019 2023-02-11 09:54:08,104 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109245396 2023-02-11 09:54:08,105 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109245396 2023-02-11 09:54:08,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,106 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109245659 2023-02-11 09:54:08,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,106 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109245925 2023-02-11 09:54:08,107 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-fa7fe2dc831a48f2a9d27b690aacd0c4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109246185 2023-02-11 09:54:08,107 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109245659 2023-02-11 09:54:08,108 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9e6964fafed1465d875cbfa140fb8c77, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109246427 2023-02-11 09:54:08,108 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=98077030a27e351168173ed0c22ff5a4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:08,108 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9338520281ae498db7acfd4210009b4e, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109246653 2023-02-11 09:54:08,108 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109248108"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109248108"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109248108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109248108"}]},"ts":"1676109248108"} 2023-02-11 09:54:08,110 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109245925 2023-02-11 09:54:08,110 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-ec260062bfe64e64bf59e377422e61bb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109246185 2023-02-11 09:54:08,112 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-e0523ccf488f438d8ac8da6f93d6ca73, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109246427 2023-02-11 09:54:08,113 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-84c382d07d654f428a78d3dd9839e7c2, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109246653 2023-02-11 09:54:08,119 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-11 09:54:08,119 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 98077030a27e351168173ed0c22ff5a4, server=jenkins-hbase4.apache.org,41949,1676109241776 in 558 msec 2023-02-11 09:54:08,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-11 09:54:08,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=98077030a27e351168173ed0c22ff5a4, ASSIGN in 728 msec 2023-02-11 09:54:08,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 in 1.4510 sec 2023-02-11 09:54:08,146 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] throttle.PressureAwareThroughputController(145): 98077030a27e351168173ed0c22ff5a4#cf1#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:08,150 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] throttle.PressureAwareThroughputController(145): 98077030a27e351168173ed0c22ff5a4#cf2#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:08,171 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/.tmp/cf1/95b3e08dc87e48d082d06d7b90401486 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/95b3e08dc87e48d082d06d7b90401486 2023-02-11 09:54:08,171 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/.tmp/cf2/f93c95dd878f4b44823c00c689b4104a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/f93c95dd878f4b44823c00c689b4104a 2023-02-11 09:54:08,179 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 98077030a27e351168173ed0c22ff5a4/cf1 of 98077030a27e351168173ed0c22ff5a4 into 95b3e08dc87e48d082d06d7b90401486(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:08,179 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 98077030a27e351168173ed0c22ff5a4: 2023-02-11 09:54:08,179 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4., storeName=98077030a27e351168173ed0c22ff5a4/cf1, priority=9, startTime=1676109248096; duration=0sec 2023-02-11 09:54:08,179 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 98077030a27e351168173ed0c22ff5a4/cf2 of 98077030a27e351168173ed0c22ff5a4 into f93c95dd878f4b44823c00c689b4104a(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:08,179 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:08,179 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 98077030a27e351168173ed0c22ff5a4: 2023-02-11 09:54:08,180 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4., storeName=98077030a27e351168173ed0c22ff5a4/cf2, priority=9, startTime=1676109248097; duration=0sec 2023-02-11 09:54:08,180 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:08,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41949] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:52822 deadline: 1676109308184, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218. is not online on jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:08,301 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 09:54:08,316 INFO [Listener at localhost/34597] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 09:54:08,319 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 09:54:08,346 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:08,356 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,373 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109248372"}]},"ts":"1676109248372"} 2023-02-11 09:54:08,375 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-11 09:54:08,378 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-11 09:54:08,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 09:54:08,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b2c7ebbb9b8d1a8598f01c1b9431d43c, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=98077030a27e351168173ed0c22ff5a4, UNASSIGN}] 2023-02-11 09:54:08,381 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=98077030a27e351168173ed0c22ff5a4, UNASSIGN 2023-02-11 09:54:08,382 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=98077030a27e351168173ed0c22ff5a4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:08,383 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109248382"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109248382"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109248382"}]},"ts":"1676109248382"} 2023-02-11 09:54:08,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 98077030a27e351168173ed0c22ff5a4, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:54:08,388 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b2c7ebbb9b8d1a8598f01c1b9431d43c, UNASSIGN 2023-02-11 09:54:08,389 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=b2c7ebbb9b8d1a8598f01c1b9431d43c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:08,389 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109248389"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109248389"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109248389"}]},"ts":"1676109248389"} 2023-02-11 09:54:08,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure b2c7ebbb9b8d1a8598f01c1b9431d43c, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:54:08,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 09:54:08,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:08,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b2c7ebbb9b8d1a8598f01c1b9431d43c, disabling compactions & flushes 2023-02-11 09:54:08,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:08,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:08,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. after waiting 0 ms 2023-02-11 09:54:08,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:08,543 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a-bottom] to archive 2023-02-11 09:54:08,544 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:08,551 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,553 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,555 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,557 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,582 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e-bottom] to archive 2023-02-11 09:54:08,583 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:08,585 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,588 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,589 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,590 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:54:08,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:08,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c. 2023-02-11 09:54:08,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b2c7ebbb9b8d1a8598f01c1b9431d43c: Waiting for close lock at 1676109248537Running coprocessor pre-close hooks at 1676109248537Disabling compacts and flushes for region at 1676109248537Disabling writes for close at 1676109248537Writing region close event to WAL at 1676109248591 (+54 ms)Running coprocessor post-close hooks at 1676109248597 (+6 ms)Closed at 1676109248597 2023-02-11 09:54:08,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:08,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:08,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 98077030a27e351168173ed0c22ff5a4, disabling compactions & flushes 2023-02-11 09:54:08,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. after waiting 0 ms 2023-02-11 09:54:08,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,600 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=b2c7ebbb9b8d1a8598f01c1b9431d43c, regionState=CLOSED 2023-02-11 09:54:08,601 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109248600"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109248600"}]},"ts":"1676109248600"} 2023-02-11 09:54:08,601 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-fa7fe2dc831a48f2a9d27b690aacd0c4, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9e6964fafed1465d875cbfa140fb8c77, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9338520281ae498db7acfd4210009b4e] to archive 2023-02-11 09:54:08,602 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:08,605 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/b527bc84bea146f79704b40b48b99dc6.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-11 09:54:08,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure b2c7ebbb9b8d1a8598f01c1b9431d43c, server=jenkins-hbase4.apache.org,41949,1676109241776 in 212 msec 2023-02-11 09:54:08,607 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/f8b40a2d8e9045549cad4b35fe58856b.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b2c7ebbb9b8d1a8598f01c1b9431d43c, UNASSIGN in 227 msec 2023-02-11 09:54:08,609 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/200a5de99e0a419186b8d1c81286386b.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,611 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/ef1d1698733b4961b6b2b3742f70261a.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,613 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-fa7fe2dc831a48f2a9d27b690aacd0c4 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-fa7fe2dc831a48f2a9d27b690aacd0c4 2023-02-11 09:54:08,614 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9e6964fafed1465d875cbfa140fb8c77 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9e6964fafed1465d875cbfa140fb8c77 2023-02-11 09:54:08,616 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9338520281ae498db7acfd4210009b4e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-9338520281ae498db7acfd4210009b4e 2023-02-11 09:54:08,618 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-ec260062bfe64e64bf59e377422e61bb, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-e0523ccf488f438d8ac8da6f93d6ca73, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-84c382d07d654f428a78d3dd9839e7c2] to archive 2023-02-11 09:54:08,620 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:08,622 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/fb42258a2612413fa71c12551940164a.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,624 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,625 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/029016d368a24b1097e79e17a9fb2819.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,627 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/09b93cb8cc6f4d429130dc66ba8a189e.d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,628 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-ec260062bfe64e64bf59e377422e61bb to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-ec260062bfe64e64bf59e377422e61bb 2023-02-11 09:54:08,630 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-e0523ccf488f438d8ac8da6f93d6ca73 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-e0523ccf488f438d8ac8da6f93d6ca73 2023-02-11 09:54:08,631 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-84c382d07d654f428a78d3dd9839e7c2 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/testSplitTableRegionUnevenDaughter=d216b600c0cb9961b975974a1b3ad218-84c382d07d654f428a78d3dd9839e7c2 2023-02-11 09:54:08,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:54:08,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:08,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4. 2023-02-11 09:54:08,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 98077030a27e351168173ed0c22ff5a4: Waiting for close lock at 1676109248600Running coprocessor pre-close hooks at 1676109248600Disabling compacts and flushes for region at 1676109248600Disabling writes for close at 1676109248600Writing region close event to WAL at 1676109248632 (+32 ms)Running coprocessor post-close hooks at 1676109248638 (+6 ms)Closed at 1676109248638 2023-02-11 09:54:08,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:08,641 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=98077030a27e351168173ed0c22ff5a4, regionState=CLOSED 2023-02-11 09:54:08,641 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676109248641"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109248641"}]},"ts":"1676109248641"} 2023-02-11 09:54:08,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-11 09:54:08,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 98077030a27e351168173ed0c22ff5a4, server=jenkins-hbase4.apache.org,41949,1676109241776 in 259 msec 2023-02-11 09:54:08,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-11 09:54:08,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=98077030a27e351168173ed0c22ff5a4, UNASSIGN in 265 msec 2023-02-11 09:54:08,649 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109248649"}]},"ts":"1676109248649"} 2023-02-11 09:54:08,650 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-11 09:54:08,652 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-11 09:54:08,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 290 msec 2023-02-11 09:54:08,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 09:54:08,686 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-11 09:54:08,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,704 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 09:54:08,708 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,717 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,718 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:08,719 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:08,728 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/recovered.edits] 2023-02-11 09:54:08,729 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/recovered.edits] 2023-02-11 09:54:08,731 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/recovered.edits] 2023-02-11 09:54:08,738 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/95b3e08dc87e48d082d06d7b90401486 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf1/95b3e08dc87e48d082d06d7b90401486 2023-02-11 09:54:08,738 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/200a5de99e0a419186b8d1c81286386b 2023-02-11 09:54:08,740 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f51691fc097d49449df1043312b5b056 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf1/f51691fc097d49449df1043312b5b056 2023-02-11 09:54:08,740 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9338520281ae498db7acfd4210009b4e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9338520281ae498db7acfd4210009b4e 2023-02-11 09:54:08,742 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9e6964fafed1465d875cbfa140fb8c77 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/9e6964fafed1465d875cbfa140fb8c77 2023-02-11 09:54:08,742 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/f93c95dd878f4b44823c00c689b4104a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/cf2/f93c95dd878f4b44823c00c689b4104a 2023-02-11 09:54:08,744 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/5e167d5bcf92453a916cd509671c2da4 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/cf2/5e167d5bcf92453a916cd509671c2da4 2023-02-11 09:54:08,748 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/b527bc84bea146f79704b40b48b99dc6 2023-02-11 09:54:08,750 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/ef1d1698733b4961b6b2b3742f70261a 2023-02-11 09:54:08,752 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4/recovered.edits/90.seqid 2023-02-11 09:54:08,754 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:08,754 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/f8b40a2d8e9045549cad4b35fe58856b 2023-02-11 09:54:08,754 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c/recovered.edits/90.seqid 2023-02-11 09:54:08,755 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/b2c7ebbb9b8d1a8598f01c1b9431d43c 2023-02-11 09:54:08,756 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf1/fa7fe2dc831a48f2a9d27b690aacd0c4 2023-02-11 09:54:08,759 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/029016d368a24b1097e79e17a9fb2819 2023-02-11 09:54:08,760 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/09b93cb8cc6f4d429130dc66ba8a189e 2023-02-11 09:54:08,762 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/37f8dbb1c42b48dfb9c10e23d2cb1ab9 2023-02-11 09:54:08,764 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/84c382d07d654f428a78d3dd9839e7c2 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/84c382d07d654f428a78d3dd9839e7c2 2023-02-11 09:54:08,765 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/e0523ccf488f438d8ac8da6f93d6ca73 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/e0523ccf488f438d8ac8da6f93d6ca73 2023-02-11 09:54:08,767 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/ec260062bfe64e64bf59e377422e61bb to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/ec260062bfe64e64bf59e377422e61bb 2023-02-11 09:54:08,768 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/cf2/fb42258a2612413fa71c12551940164a 2023-02-11 09:54:08,771 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/recovered.edits/85.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218/recovered.edits/85.seqid 2023-02-11 09:54:08,772 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionUnevenDaughter/d216b600c0cb9961b975974a1b3ad218 2023-02-11 09:54:08,772 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 09:54:08,775 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,798 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-11 09:54:08,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 09:54:08,839 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-11 09:54:08,841 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,841 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-11 09:54:08,841 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109248841"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:08,841 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109248841"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:08,841 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109248841"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:08,844 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 09:54:08,844 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d216b600c0cb9961b975974a1b3ad218, NAME => 'testSplitTableRegionUnevenDaughter,,1676109244503.d216b600c0cb9961b975974a1b3ad218.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b2c7ebbb9b8d1a8598f01c1b9431d43c, NAME => 'testSplitTableRegionUnevenDaughter,,1676109246674.b2c7ebbb9b8d1a8598f01c1b9431d43c.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 98077030a27e351168173ed0c22ff5a4, NAME => 'testSplitTableRegionUnevenDaughter,26,1676109246674.98077030a27e351168173ed0c22ff5a4.', STARTKEY => '26', ENDKEY => ''}] 2023-02-11 09:54:08,844 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-11 09:54:08,844 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109248844"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:08,846 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-11 09:54:08,850 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:08,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 157 msec 2023-02-11 09:54:08,973 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:54:09,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 09:54:09,011 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-11 09:54:09,032 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=392 (was 359) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:54906 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6a75bb5c-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:60662 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:50196 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:50354 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:60478 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6a75bb5c-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:50348 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:54892 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:54614 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:34198 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:54576 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:41949-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:54646 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:46346 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6a75bb5c-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:60670 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:60650 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:50332 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_842581441_15 at /127.0.0.1:50368 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=851 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=313 (was 314), ProcessCount=169 (was 169), AvailableMemoryMB=3036 (was 3103) 2023-02-11 09:54:09,041 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-11 09:54:09,042 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-11 09:54:09,048 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=392, OpenFileDescriptor=851, MaxFileDescriptor=60000, SystemLoadAverage=313, ProcessCount=169, AvailableMemoryMB=3034 2023-02-11 09:54:09,048 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:09,054 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:54:09,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:54:09,057 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:09,059 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:54:09,060 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:54:09,062 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,062 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8 empty. 2023-02-11 09:54:09,063 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,063 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 09:54:09,075 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:09,076 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4b63fe694a11cba94c671f4e6eff38f8, NAME => 'testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:54:09,088 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:09,088 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 4b63fe694a11cba94c671f4e6eff38f8, disabling compactions & flushes 2023-02-11 09:54:09,088 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,088 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,088 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. after waiting 0 ms 2023-02-11 09:54:09,088 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,088 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,088 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 4b63fe694a11cba94c671f4e6eff38f8: Waiting for close lock at 1676109249088Disabling compacts and flushes for region at 1676109249088Disabling writes for close at 1676109249088Writing region close event to WAL at 1676109249088Closed at 1676109249088 2023-02-11 09:54:09,091 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:54:09,092 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109249092"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109249092"}]},"ts":"1676109249092"} 2023-02-11 09:54:09,094 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:54:09,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:54:09,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109249095"}]},"ts":"1676109249095"} 2023-02-11 09:54:09,097 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-11 09:54:09,101 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:09,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:09,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:09,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:09,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:09,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, ASSIGN}] 2023-02-11 09:54:09,106 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, ASSIGN 2023-02-11 09:54:09,107 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=false 2023-02-11 09:54:09,257 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:09,258 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4b63fe694a11cba94c671f4e6eff38f8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,259 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109249258"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109249258"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109249258"}]},"ts":"1676109249258"} 2023-02-11 09:54:09,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:09,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4b63fe694a11cba94c671f4e6eff38f8, NAME => 'testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:09,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:09,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:09,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,419 INFO [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,420 DEBUG [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1 2023-02-11 09:54:09,420 DEBUG [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1 2023-02-11 09:54:09,421 INFO [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b63fe694a11cba94c671f4e6eff38f8 columnFamilyName cf1 2023-02-11 09:54:09,421 INFO [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] regionserver.HStore(310): Store=4b63fe694a11cba94c671f4e6eff38f8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:09,422 INFO [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,423 DEBUG [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2 2023-02-11 09:54:09,423 DEBUG [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2 2023-02-11 09:54:09,424 INFO [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b63fe694a11cba94c671f4e6eff38f8 columnFamilyName cf2 2023-02-11 09:54:09,425 INFO [StoreOpener-4b63fe694a11cba94c671f4e6eff38f8-1] regionserver.HStore(310): Store=4b63fe694a11cba94c671f4e6eff38f8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:09,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:09,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:09,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:09,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4b63fe694a11cba94c671f4e6eff38f8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66966511, jitterRate=-0.0021212249994277954}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:09,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4b63fe694a11cba94c671f4e6eff38f8: Running coprocessor pre-open hook at 1676109249418Writing region info on filesystem at 1676109249418Initializing all the Stores at 1676109249418Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109249419 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109249419Cleaning up temporary data from old regions at 1676109249427 (+8 ms)Cleaning up detritus from prior splits at 1676109249427Running coprocessor post-open hooks at 1676109249432 (+5 ms)Region opened successfully at 1676109249432 2023-02-11 09:54:09,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8., pid=27, masterSystemTime=1676109249413 2023-02-11 09:54:09,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,436 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4b63fe694a11cba94c671f4e6eff38f8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,436 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109249436"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109249436"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109249436"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109249436"}]},"ts":"1676109249436"} 2023-02-11 09:54:09,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-11 09:54:09,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 in 177 msec 2023-02-11 09:54:09,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-11 09:54:09,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, ASSIGN in 338 msec 2023-02-11 09:54:09,445 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:54:09,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109249445"}]},"ts":"1676109249445"} 2023-02-11 09:54:09,446 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-11 09:54:09,449 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:54:09,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 394 msec 2023-02-11 09:54:09,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:09,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 09:54:09,570 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 09:54:09,570 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,571 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 09:54:09,571 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 09:54:09,572 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,572 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,574 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,574 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,574 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,574 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,575 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,575 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,580 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,581 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. started... 2023-02-11 09:54:09,582 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b63fe694a11cba94c671f4e6eff38f8 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:54:09,583 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,584 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,584 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,584 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,584 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,584 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,595 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/76e8ffb2449b40ee8b8908ecded94db2 2023-02-11 09:54:09,611 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/3201634bc3c240e890bf8bb97b13230e 2023-02-11 09:54:09,616 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/76e8ffb2449b40ee8b8908ecded94db2 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2 2023-02-11 09:54:09,621 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:09,622 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/3201634bc3c240e890bf8bb97b13230e as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e 2023-02-11 09:54:09,626 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:09,627 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 45ms, sequenceid=6, compaction requested=false 2023-02-11 09:54:09,627 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-11 09:54:09,628 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b63fe694a11cba94c671f4e6eff38f8: 2023-02-11 09:54:09,628 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,630 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:09,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 09:54:09,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4be11fd0[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:09,632 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 09:54:09,632 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,634 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,634 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,634 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,635 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:09,635 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:09,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:09,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:09,637 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,637 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:09,638 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,637 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:09,638 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:09,638 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,638 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:09,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 09:54:09,642 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 09:54:09,642 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:09,642 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:09,642 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 09:54:09,642 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:09,643 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,643 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,644 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,643 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:09,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,644 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,644 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,645 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,645 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:09,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,647 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:09,647 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,647 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 09:54:09,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:09,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,652 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,653 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:09,653 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,653 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,653 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,653 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,654 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,753 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:09,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:09,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:09,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:09,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 09:54:09,774 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 09:54:09,774 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,775 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 09:54:09,775 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 09:54:09,776 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,776 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,777 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,777 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,777 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,777 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,777 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,777 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,779 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,779 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,779 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,779 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,781 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,781 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,781 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,781 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,782 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,780 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,782 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,782 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. started... 2023-02-11 09:54:09,782 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b63fe694a11cba94c671f4e6eff38f8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:09,783 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,794 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/1dc77dfaefdc40e6982f661c6e7f324e 2023-02-11 09:54:09,808 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/6b5c4f9ec56a4e2faef244445e05ed42 2023-02-11 09:54:09,813 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/1dc77dfaefdc40e6982f661c6e7f324e as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e 2023-02-11 09:54:09,818 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:09,819 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/6b5c4f9ec56a4e2faef244445e05ed42 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42 2023-02-11 09:54:09,823 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:09,824 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 42ms, sequenceid=19, compaction requested=false 2023-02-11 09:54:09,824 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b63fe694a11cba94c671f4e6eff38f8: 2023-02-11 09:54:09,824 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,824 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:09,824 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:09,824 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,824 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,824 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,827 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,827 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:09,827 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,827 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:09,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:09,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:09,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 09:54:09,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d4ca3c3[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:09,829 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 09:54:09,829 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,830 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,830 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,830 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:09,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:09,831 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,831 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:09,831 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:09,832 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,832 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:09,833 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:09,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:09,833 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:09,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:09,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:09,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:09,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:09,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 09:54:09,842 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 09:54:09,842 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:09,842 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:09,842 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 09:54:09,842 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:09,844 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:09,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,845 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,844 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:09,845 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,846 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,848 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:09,848 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:09,849 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,851 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,851 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,851 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,851 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,851 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,851 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,852 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:09,852 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 09:54:09,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:09,852 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:09,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,852 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:09,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:09,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,853 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:09,854 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,854 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:09,954 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:09,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:09,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:09,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:09,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 09:54:09,974 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 09:54:09,974 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,975 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 09:54:09,975 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 09:54:09,976 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,976 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,978 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,978 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,978 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,978 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:09,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,978 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:09,978 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:09,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:09,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:09,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:09,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:09,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:09,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:09,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:09,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:09,980 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,980 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,980 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,980 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:09,981 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,981 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,981 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,981 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,980 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:09,981 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,982 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:09,981 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:09,982 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:09,982 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:09,982 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:09,982 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. started... 2023-02-11 09:54:09,982 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:09,982 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b63fe694a11cba94c671f4e6eff38f8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:09,995 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/6498f3473b174284b4a2351c50e3cbf2 2023-02-11 09:54:10,009 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/021f2c69ab274d7484aa46fa75a371e9 2023-02-11 09:54:10,015 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/6498f3473b174284b4a2351c50e3cbf2 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2 2023-02-11 09:54:10,021 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:10,022 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/021f2c69ab274d7484aa46fa75a371e9 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9 2023-02-11 09:54:10,027 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:10,028 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 46ms, sequenceid=32, compaction requested=true 2023-02-11 09:54:10,028 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b63fe694a11cba94c671f4e6eff38f8: 2023-02-11 09:54:10,028 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,030 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,030 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,030 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,030 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 09:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6aa53435[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:10,032 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 09:54:10,032 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,035 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,035 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:10,036 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,036 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,037 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,038 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,038 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,038 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,038 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,039 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,039 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,039 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:10,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 09:54:10,042 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 09:54:10,042 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:10,042 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:10,042 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 09:54:10,042 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:10,044 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,044 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,044 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,045 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,046 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,048 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,050 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,050 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,050 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,053 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,053 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,053 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,053 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,053 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,053 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,053 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,053 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,054 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 09:54:10,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:10,054 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,054 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,155 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:10,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:10,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:10,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:10,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 09:54:10,179 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 09:54:10,179 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,179 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 09:54:10,179 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 09:54:10,180 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,180 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,181 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,181 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,181 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,181 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,182 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,182 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:10,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,183 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,183 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,183 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,184 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,184 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,184 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,184 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,183 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,185 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,185 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,186 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,186 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,186 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,187 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:10,187 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:10,187 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:10,187 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. started... 2023-02-11 09:54:10,187 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b63fe694a11cba94c671f4e6eff38f8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:10,194 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,195 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,195 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,195 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,196 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,196 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,202 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/7de12515aebb4f84b9b8b6da454e3af7 2023-02-11 09:54:10,216 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 2023-02-11 09:54:10,221 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/7de12515aebb4f84b9b8b6da454e3af7 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7 2023-02-11 09:54:10,226 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:10,227 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 2023-02-11 09:54:10,231 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:10,232 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 45ms, sequenceid=45, compaction requested=true 2023-02-11 09:54:10,233 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b63fe694a11cba94c671f4e6eff38f8: 2023-02-11 09:54:10,233 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:10,233 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:10,233 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:10,233 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,233 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,233 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,235 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,235 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,235 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,235 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 09:54:10,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7161ce1[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:10,239 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 09:54:10,239 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,241 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,241 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,241 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,242 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,242 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:10,242 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:10,244 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,244 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,244 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,245 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,245 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,245 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,245 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,245 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,246 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,246 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:10,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 09:54:10,252 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 09:54:10,253 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:10,253 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:10,253 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 09:54:10,253 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:10,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,256 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,257 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,257 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,259 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,259 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,260 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,262 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,262 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,262 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,262 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,262 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,262 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 09:54:10,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:10,262 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,263 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,263 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,263 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,263 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,263 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:10,263 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,263 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:10,263 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,364 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:10,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:10,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:10,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:10,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 09:54:10,386 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 09:54:10,386 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,386 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 09:54:10,386 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 09:54:10,386 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,386 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,388 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,388 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,388 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,388 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,388 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,388 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:10,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,390 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,390 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,390 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,391 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:10,391 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:10,391 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. started... 2023-02-11 09:54:10,392 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b63fe694a11cba94c671f4e6eff38f8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:10,392 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,395 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,395 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,410 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/711fbfa50895401a8bda4fc448afd8ca 2023-02-11 09:54:10,425 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/05fc5772e02d46c3aa3a967c52f26789 2023-02-11 09:54:10,431 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/711fbfa50895401a8bda4fc448afd8ca as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca 2023-02-11 09:54:10,436 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:10,437 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/05fc5772e02d46c3aa3a967c52f26789 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789 2023-02-11 09:54:10,443 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:10,444 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 53ms, sequenceid=58, compaction requested=true 2023-02-11 09:54:10,444 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b63fe694a11cba94c671f4e6eff38f8: 2023-02-11 09:54:10,444 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:10,444 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:10,444 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:10,444 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,444 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,444 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,446 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,446 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,447 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,447 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 09:54:10,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@444a8c4c[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:10,449 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 09:54:10,449 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,450 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:10,450 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:10,452 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,452 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,452 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,453 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,453 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,453 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,453 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:10,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 09:54:10,457 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 09:54:10,457 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:10,457 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:10,457 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 09:54:10,457 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:10,458 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,458 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,458 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,459 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,460 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,460 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,462 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,463 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,463 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,469 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 09:54:10,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,470 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:10,470 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:10,570 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:10,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:10,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:10,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:10,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 09:54:10,594 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 09:54:10,594 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,594 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 09:54:10,594 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 09:54:10,594 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,595 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,596 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,596 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,596 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,596 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,597 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,597 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:10,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 09:54:10,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,598 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,598 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,598 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:10,599 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:10,599 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. started... 2023-02-11 09:54:10,599 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b63fe694a11cba94c671f4e6eff38f8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:10,600 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,602 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,602 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,602 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,611 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/fb527345f1f84478b2b9ad4a8ec1991e 2023-02-11 09:54:10,626 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/a688546bb00a4aa2ab046ef7500bcc3c 2023-02-11 09:54:10,633 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/fb527345f1f84478b2b9ad4a8ec1991e as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e 2023-02-11 09:54:10,639 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:10,640 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/a688546bb00a4aa2ab046ef7500bcc3c as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c 2023-02-11 09:54:10,646 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:10,647 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 48ms, sequenceid=71, compaction requested=true 2023-02-11 09:54:10,647 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b63fe694a11cba94c671f4e6eff38f8: 2023-02-11 09:54:10,647 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 09:54:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 09:54:10,649 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,649 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,649 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,649 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 09:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cd07dbd[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:10,652 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 09:54:10,652 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,653 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,653 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,653 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:10,654 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,654 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 09:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,656 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 09:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 09:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:10,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 09:54:10,662 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 09:54:10,662 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:10,662 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:10,662 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 09:54:10,662 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:10,664 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:10,664 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,664 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:10,665 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,666 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,666 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,669 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:10,669 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 09:54:10,670 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,673 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:10,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:10,673 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,673 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 09:54:10,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,674 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:10,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:10,674 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 09:54:10,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 09:54:10,708 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 09:54:10,708 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-11 09:54:10,709 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:10,709 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-11 09:54:10,775 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:10,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:11,040 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 09:54:11,041 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:11,042 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,043 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,043 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,043 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:11,044 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:11,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:11,292 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,294 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:11,294 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:11,294 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:11,296 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-11 09:54:11,296 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:11,296 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:11,297 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:11,297 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:11,297 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:11,301 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 09:54:11,302 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:11,303 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:11,303 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:11,303 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:11,303 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,304 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:11,304 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,304 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,305 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:11,305 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,306 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:11,306 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,306 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,306 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,306 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-11 09:54:11,307 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:11,307 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:11,308 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109281308 2023-02-11 09:54:11,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,310 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:11,313 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:11,314 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:11,314 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:11,314 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109311314 2023-02-11 09:54:11,314 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109371314 2023-02-11 09:54:11,314 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 09:54:11,314 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:11,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:11,316 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:11,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:11,564 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,564 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:11,564 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:11,565 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:11,566 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:11,566 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:11,566 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:11,566 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:11,566 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:11,567 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:11,568 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 09:54:11,569 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:11,569 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:11,569 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:11,569 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:11,570 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,570 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:11,570 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,571 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,571 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:11,571 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,571 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:11,572 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,572 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,572 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,572 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:11,572 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:11,573 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:11,573 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109281573 2023-02-11 09:54:11,575 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,575 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:11,577 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:11,577 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:11,578 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:11,578 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109311578 2023-02-11 09:54:11,578 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109371578 2023-02-11 09:54:11,578 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:11,578 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:11,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN}] 2023-02-11 09:54:11,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:11,580 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:11,581 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:11,828 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,828 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:11,828 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:11,828 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:11,829 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:11,829 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:11,829 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:11,829 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:11,829 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:11,830 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:11,831 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 09:54:11,831 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:11,832 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:11,832 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:11,832 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:11,833 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,833 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:11,834 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,834 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:11,834 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:11,835 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,835 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:11,835 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,835 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,835 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,835 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:11,836 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:11,836 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:11,836 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109281836 2023-02-11 09:54:11,838 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:11,838 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:11,841 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:11,841 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:11,841 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:11,841 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109311841 2023-02-11 09:54:11,841 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109371841 2023-02-11 09:54:11,841 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:11,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN}] 2023-02-11 09:54:11,842 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:11,843 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:11,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:11,845 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109251844"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109251844"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109251844"}]},"ts":"1676109251844"} 2023-02-11 09:54:11,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:11,846 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:11,846 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:11,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:11,846 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:12,091 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,092 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:12,092 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:12,092 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:12,092 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:12,092 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:12,092 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:12,092 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:12,092 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:12,093 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:12,094 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:12,094 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 09:54:12,095 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:12,095 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:12,095 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:12,096 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:12,096 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,097 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:12,097 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,097 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,097 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:12,098 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,098 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,098 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:12,098 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,098 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,099 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,099 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:12,099 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,099 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:12,099 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:12,100 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:12,100 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8 to restore RIT 2023-02-11 09:54:12,100 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109282100 2023-02-11 09:54:12,101 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:12,101 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,104 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,104 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,104 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:12,104 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109312104 2023-02-11 09:54:12,104 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109372104 2023-02-11 09:54:12,105 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:12,105 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:12,105 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109252105"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109252105"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109252105"}]},"ts":"1676109252105"} 2023-02-11 09:54:12,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:12,107 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:12,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4b63fe694a11cba94c671f4e6eff38f8, disabling compactions & flushes 2023-02-11 09:54:12,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:12,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:12,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. after waiting 0 ms 2023-02-11 09:54:12,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:12,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 4b63fe694a11cba94c671f4e6eff38f8 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 09:54:12,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 2023-02-11 09:54:12,291 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/66666b1be8e8458a951ec423372b7f6b 2023-02-11 09:54:12,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 2023-02-11 09:54:12,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:54:12,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/.tmp/cf2/66666b1be8e8458a951ec423372b7f6b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b 2023-02-11 09:54:12,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:54:12,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 49ms, sequenceid=82, compaction requested=true 2023-02-11 09:54:12,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 09:54:12,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:12,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. 2023-02-11 09:54:12,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4b63fe694a11cba94c671f4e6eff38f8: Waiting for close lock at 1676109252260Running coprocessor pre-close hooks at 1676109252260Disabling compacts and flushes for region at 1676109252260Disabling writes for close at 1676109252260Obtaining lock to block concurrent updates at 1676109252260Preparing flush snapshotting stores in 4b63fe694a11cba94c671f4e6eff38f8 at 1676109252260Finished memstore snapshotting testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676109252261 (+1 ms)Flushing stores of testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. at 1676109252261Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf1: creating writer at 1676109252261Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf1: appending metadata at 1676109252264 (+3 ms)Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf1: closing flushed file at 1676109252264Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf2: creating writer at 1676109252279 (+15 ms)Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf2: appending metadata at 1676109252281 (+2 ms)Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf2: closing flushed file at 1676109252281Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf1: reopening flushed file at 1676109252298 (+17 ms)Flushing 4b63fe694a11cba94c671f4e6eff38f8/cf2: reopening flushed file at 1676109252304 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4b63fe694a11cba94c671f4e6eff38f8 in 49ms, sequenceid=82, compaction requested=true at 1676109252310 (+6 ms)Writing region close event to WAL at 1676109252325 (+15 ms)Running coprocessor post-close hooks at 1676109252329 (+4 ms)Closed at 1676109252329 2023-02-11 09:54:12,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,333 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED 2023-02-11 09:54:12,333 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109252333"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109252333"}]},"ts":"1676109252333"} 2023-02-11 09:54:12,334 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:12,334 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:12,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:12,335 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:12,355 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,355 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:12,355 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:12,355 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:12,356 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:12,356 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:12,356 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:12,356 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:12,356 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:12,357 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:12,358 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:12,358 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 09:54:12,358 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 09:54:12,358 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:12,359 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:12,359 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:12,359 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:12,359 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,360 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:12,360 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,360 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,360 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:12,361 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,361 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,361 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:12,361 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:12,361 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,362 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,362 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,362 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:12,362 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,362 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:12,362 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:12,362 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:12,363 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:12,363 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8 to restore RIT 2023-02-11 09:54:12,363 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109282363 2023-02-11 09:54:12,364 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:12,364 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:12,366 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,366 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,366 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:12,366 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109312366 2023-02-11 09:54:12,366 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109372366 2023-02-11 09:54:12,367 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:54:12,367 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:12,367 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109252367"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109252367"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109252367"}]},"ts":"1676109252367"} 2023-02-11 09:54:12,368 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:12,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-11 09:54:12,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 in 261 msec 2023-02-11 09:54:12,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:12,371 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:12,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:12,617 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,617 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:12,617 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:12,617 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:12,618 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:12,618 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:12,618 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:12,618 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:12,618 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:12,619 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:12,620 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:12,621 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 09:54:12,621 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:12,621 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:12,622 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:12,622 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:12,622 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:12,622 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,623 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:12,623 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,623 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,624 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:12,624 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,624 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,624 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:12,624 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:12,625 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,625 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,625 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,625 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:12,625 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:12,625 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:12,626 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:12,626 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:12,626 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8 to restore RIT 2023-02-11 09:54:12,626 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109282626 2023-02-11 09:54:12,627 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:12,627 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,630 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,630 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,630 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:12,630 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109312630 2023-02-11 09:54:12,630 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109372630 2023-02-11 09:54:12,630 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:12,630 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:12,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-11 09:54:12,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN in 791 msec 2023-02-11 09:54:12,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:12,638 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:12,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:12,880 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,881 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:12,881 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:12,881 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:12,881 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:12,881 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:12,881 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:12,881 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:12,881 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:12,882 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:12,883 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:12,883 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:12,883 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:12,884 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:12,884 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:12,884 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:12,885 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:12,885 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,885 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:12,886 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,886 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:12,886 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:12,887 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,887 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:12,887 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:12,887 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:12,887 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:12,887 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:12,887 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:12,888 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:12,888 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:12,889 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109282889 2023-02-11 09:54:12,890 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:12,890 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,892 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,893 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:12,893 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:12,893 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109312893 2023-02-11 09:54:12,893 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109372893 2023-02-11 09:54:12,893 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:54:12,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:12,898 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:12,908 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4b63fe694a11cba94c671f4e6eff38f8, threads=14 2023-02-11 09:54:12,909 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,909 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,909 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,909 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,909 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,909 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,910 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,910 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,910 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,910 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,912 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,912 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,912 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:12,913 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,016 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,017 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,034 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,035 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,069 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,080 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,083 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,084 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,085 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,087 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,089 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,089 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,090 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,438 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,438 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4b63fe694a11cba94c671f4e6eff38f8 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:54:13,485 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:13,485 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:13,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:13,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:13,643 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:13,643 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:13,644 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:13,644 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:13,644 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:13,644 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:13,644 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:13,644 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:13,644 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:13,645 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:13,646 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 09:54:13,646 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:13,646 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:13,647 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:13,647 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:13,647 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:13,648 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:13,648 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:13,648 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:13,649 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:13,649 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:13,649 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:13,650 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:13,650 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:13,650 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:13,650 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:13,651 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:13,651 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:13,651 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:13,651 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:13,651 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:13,651 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:13,651 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:13,652 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:13,652 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109283652 2023-02-11 09:54:13,653 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:13,654 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:13,656 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:13,657 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:13,657 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:13,657 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109313657 2023-02-11 09:54:13,657 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109373657 2023-02-11 09:54:13,657 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:13,667 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4b63fe694a11cba94c671f4e6eff38f8, threads=14 2023-02-11 09:54:13,668 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,668 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,668 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,668 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,668 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,669 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,670 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,670 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,670 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,670 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,670 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,670 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,670 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,674 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,828 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,835 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,835 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,845 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,845 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,850 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,853 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,856 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,861 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:13,864 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:14,099 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:54:14,218 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:14,249 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2 for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:14,257 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:14,260 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e for region: 4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:14,261 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4b63fe694a11cba94c671f4e6eff38f8 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:54:14,306 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:14,312 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:54:14,314 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:54:14,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:14,315 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:14,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:14,407 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,407 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:14,407 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:14,408 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:14,409 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:14,409 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:14,409 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:14,409 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:14,409 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:14,409 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:14,411 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 09:54:14,411 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:14,411 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:14,411 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:14,412 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:14,412 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:14,412 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:14,412 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:14,413 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:14,413 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:14,413 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:14,413 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:14,414 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,414 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:14,414 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:14,414 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:14,414 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,414 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,414 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,414 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:14,415 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:14,415 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:14,415 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:14,416 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:14,416 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109284416 2023-02-11 09:54:14,417 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:14,417 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,420 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:14,420 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:14,420 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:14,420 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109314420 2023-02-11 09:54:14,420 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109374420 2023-02-11 09:54:14,420 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:14,423 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:14,425 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:14,425 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:14,425 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:14,670 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:14,671 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:14,671 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:14,671 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:14,671 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:14,671 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:14,671 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:14,671 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:14,671 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:14,672 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:14,673 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 09:54:14,673 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:14,673 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:14,673 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:14,674 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:14,674 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:14,674 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:14,674 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:14,675 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:14,675 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:14,675 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:14,675 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:14,676 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,676 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN 2023-02-11 09:54:14,676 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:14,676 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:14,676 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,676 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,677 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,677 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b63fe694a11cba94c671f4e6eff38f8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:14,677 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4b63fe694a11cba94c671f4e6eff38f8, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:14,677 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 09:54:14,677 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:14,678 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:14,678 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109284678 2023-02-11 09:54:14,679 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:14,679 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:14,681 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:14,682 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=4b63fe694a11cba94c671f4e6eff38f8, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:14,682 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:14,682 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109314682 2023-02-11 09:54:14,682 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109374682 2023-02-11 09:54:14,682 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:14,684 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109254684"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676109254684"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676109254684"}]},"ts":"1676109254684"} 2023-02-11 09:54:14,684 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109254684"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109254684"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109254684"}]},"ts":"1676109254684"} 2023-02-11 09:54:14,684 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109254684"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109254684"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109254684"}]},"ts":"1676109254684"} 2023-02-11 09:54:14,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a7bf2dbceab20d06148a05aabdce5a38, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=846a3db876efcaea894eafcf63ddffd7, ASSIGN}] 2023-02-11 09:54:14,691 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=846a3db876efcaea894eafcf63ddffd7, ASSIGN 2023-02-11 09:54:14,692 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=846a3db876efcaea894eafcf63ddffd7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=false 2023-02-11 09:54:14,692 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a7bf2dbceab20d06148a05aabdce5a38, ASSIGN 2023-02-11 09:54:14,693 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a7bf2dbceab20d06148a05aabdce5a38, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=false 2023-02-11 09:54:14,842 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 09:54:14,843 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=a7bf2dbceab20d06148a05aabdce5a38, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:14,844 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109254843"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109254843"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109254843"}]},"ts":"1676109254843"} 2023-02-11 09:54:14,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure a7bf2dbceab20d06148a05aabdce5a38, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:14,849 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=846a3db876efcaea894eafcf63ddffd7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:14,849 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109254849"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109254849"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109254849"}]},"ts":"1676109254849"} 2023-02-11 09:54:14,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 846a3db876efcaea894eafcf63ddffd7, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:15,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 846a3db876efcaea894eafcf63ddffd7, NAME => 'testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 09:54:15,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:15,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:15,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,003 INFO [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,005 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1 2023-02-11 09:54:15,005 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1 2023-02-11 09:54:15,005 INFO [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 846a3db876efcaea894eafcf63ddffd7 columnFamilyName cf1 2023-02-11 09:54:15,022 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e-top 2023-02-11 09:54:15,026 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2-top 2023-02-11 09:54:15,030 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca-top 2023-02-11 09:54:15,034 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60-top 2023-02-11 09:54:15,039 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2-top 2023-02-11 09:54:15,043 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7-top 2023-02-11 09:54:15,047 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e-top 2023-02-11 09:54:15,047 INFO [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(310): Store=846a3db876efcaea894eafcf63ddffd7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:15,047 INFO [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,048 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2 2023-02-11 09:54:15,048 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2 2023-02-11 09:54:15,049 INFO [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 846a3db876efcaea894eafcf63ddffd7 columnFamilyName cf2 2023-02-11 09:54:15,062 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9-top 2023-02-11 09:54:15,066 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789-top 2023-02-11 09:54:15,070 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e-top 2023-02-11 09:54:15,075 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b-top 2023-02-11 09:54:15,080 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42-top 2023-02-11 09:54:15,085 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c-top 2023-02-11 09:54:15,090 DEBUG [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5-top 2023-02-11 09:54:15,090 INFO [StoreOpener-846a3db876efcaea894eafcf63ddffd7-1] regionserver.HStore(310): Store=846a3db876efcaea894eafcf63ddffd7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:15,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:15,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 846a3db876efcaea894eafcf63ddffd7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72386715, jitterRate=0.07864610850811005}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:15,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 846a3db876efcaea894eafcf63ddffd7: Running coprocessor pre-open hook at 1676109255002Writing region info on filesystem at 1676109255002Initializing all the Stores at 1676109255003 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109255003Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109255003Cleaning up temporary data from old regions at 1676109255092 (+89 ms)Cleaning up detritus from prior splits at 1676109255093 (+1 ms)Running coprocessor post-open hooks at 1676109255096 (+3 ms)Region opened successfully at 1676109255096 2023-02-11 09:54:15,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7., pid=34, masterSystemTime=1676109254998 2023-02-11 09:54:15,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:15,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:15,098 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:15,098 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:15,098 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,098 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1912): 846a3db876efcaea894eafcf63ddffd7/cf2 is initiating minor compaction (all files) 2023-02-11 09:54:15,098 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,098 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 846a3db876efcaea894eafcf63ddffd7/cf2 in testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,098 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1912): 846a3db876efcaea894eafcf63ddffd7/cf1 is initiating minor compaction (all files) 2023-02-11 09:54:15,099 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 846a3db876efcaea894eafcf63ddffd7/cf1 in testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,099 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b-top] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/.tmp, totalSize=34.9 K 2023-02-11 09:54:15,099 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60-top] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/.tmp, totalSize=34.9 K 2023-02-11 09:54:15,099 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109249561 2023-02-11 09:54:15,100 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109249561 2023-02-11 09:54:15,100 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109249755 2023-02-11 09:54:15,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a7bf2dbceab20d06148a05aabdce5a38, NAME => 'testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 09:54:15,100 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109249755 2023-02-11 09:54:15,101 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109249956 2023-02-11 09:54:15,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:15,101 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109249956 2023-02-11 09:54:15,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:15,101 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109250157 2023-02-11 09:54:15,101 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109250157 2023-02-11 09:54:15,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,102 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676109250366 2023-02-11 09:54:15,102 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676109250366 2023-02-11 09:54:15,102 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=846a3db876efcaea894eafcf63ddffd7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:15,102 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676109250574 2023-02-11 09:54:15,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109255102"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109255102"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109255102"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109255102"}]},"ts":"1676109255102"} 2023-02-11 09:54:15,103 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676109250574 2023-02-11 09:54:15,104 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109250777 2023-02-11 09:54:15,104 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109250777 2023-02-11 09:54:15,106 INFO [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,107 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1 2023-02-11 09:54:15,108 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1 2023-02-11 09:54:15,108 INFO [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7bf2dbceab20d06148a05aabdce5a38 columnFamilyName cf1 2023-02-11 09:54:15,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-11 09:54:15,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 846a3db876efcaea894eafcf63ddffd7, server=jenkins-hbase4.apache.org,34799,1676109241745 in 255 msec 2023-02-11 09:54:15,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=846a3db876efcaea894eafcf63ddffd7, ASSIGN in 421 msec 2023-02-11 09:54:15,124 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] throttle.PressureAwareThroughputController(145): 846a3db876efcaea894eafcf63ddffd7#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:15,124 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] throttle.PressureAwareThroughputController(145): 846a3db876efcaea894eafcf63ddffd7#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:15,132 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e-bottom 2023-02-11 09:54:15,136 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2-bottom 2023-02-11 09:54:15,140 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca-bottom 2023-02-11 09:54:15,148 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60-bottom 2023-02-11 09:54:15,155 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2-bottom 2023-02-11 09:54:15,164 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/.tmp/cf2/529b25f3141048399d717673926c5545 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/529b25f3141048399d717673926c5545 2023-02-11 09:54:15,165 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7-bottom 2023-02-11 09:54:15,169 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/.tmp/cf1/2ca29bafd40945bbb1efed9dec906021 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/2ca29bafd40945bbb1efed9dec906021 2023-02-11 09:54:15,170 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e-bottom 2023-02-11 09:54:15,170 INFO [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(310): Store=a7bf2dbceab20d06148a05aabdce5a38/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:15,170 INFO [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,171 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 846a3db876efcaea894eafcf63ddffd7/cf2 of 846a3db876efcaea894eafcf63ddffd7 into 529b25f3141048399d717673926c5545(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:15,172 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 846a3db876efcaea894eafcf63ddffd7: 2023-02-11 09:54:15,172 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7., storeName=846a3db876efcaea894eafcf63ddffd7/cf2, priority=9, startTime=1676109255097; duration=0sec 2023-02-11 09:54:15,172 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:15,173 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2 2023-02-11 09:54:15,173 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2 2023-02-11 09:54:15,173 INFO [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7bf2dbceab20d06148a05aabdce5a38 columnFamilyName cf2 2023-02-11 09:54:15,178 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 846a3db876efcaea894eafcf63ddffd7/cf1 of 846a3db876efcaea894eafcf63ddffd7 into 2ca29bafd40945bbb1efed9dec906021(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:15,178 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 846a3db876efcaea894eafcf63ddffd7: 2023-02-11 09:54:15,178 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7., storeName=846a3db876efcaea894eafcf63ddffd7/cf1, priority=9, startTime=1676109255097; duration=0sec 2023-02-11 09:54:15,178 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:15,189 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9-bottom 2023-02-11 09:54:15,193 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789-bottom 2023-02-11 09:54:15,198 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e-bottom 2023-02-11 09:54:15,202 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b-bottom 2023-02-11 09:54:15,205 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42-bottom 2023-02-11 09:54:15,210 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c-bottom 2023-02-11 09:54:15,214 DEBUG [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5-bottom 2023-02-11 09:54:15,214 INFO [StoreOpener-a7bf2dbceab20d06148a05aabdce5a38-1] regionserver.HStore(310): Store=a7bf2dbceab20d06148a05aabdce5a38/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:15,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:15,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a7bf2dbceab20d06148a05aabdce5a38; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60857288, jitterRate=-0.09315574169158936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:15,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a7bf2dbceab20d06148a05aabdce5a38: Running coprocessor pre-open hook at 1676109255102Writing region info on filesystem at 1676109255102Initializing all the Stores at 1676109255104 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109255105 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109255106 (+1 ms)Cleaning up temporary data from old regions at 1676109255217 (+111 ms)Cleaning up detritus from prior splits at 1676109255217Running coprocessor post-open hooks at 1676109255220 (+3 ms)Region opened successfully at 1676109255220 2023-02-11 09:54:15,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38., pid=33, masterSystemTime=1676109254998 2023-02-11 09:54:15,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:15,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:15,221 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:15,221 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:15,222 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,222 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,222 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1912): a7bf2dbceab20d06148a05aabdce5a38/cf1 is initiating minor compaction (all files) 2023-02-11 09:54:15,222 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1912): a7bf2dbceab20d06148a05aabdce5a38/cf2 is initiating minor compaction (all files) 2023-02-11 09:54:15,222 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a7bf2dbceab20d06148a05aabdce5a38/cf1 in testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,222 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a7bf2dbceab20d06148a05aabdce5a38/cf2 in testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,222 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/.tmp, totalSize=34.9 K 2023-02-11 09:54:15,222 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/.tmp, totalSize=34.9 K 2023-02-11 09:54:15,223 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109249561 2023-02-11 09:54:15,223 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109249561 2023-02-11 09:54:15,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,224 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109249755 2023-02-11 09:54:15,224 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109249755 2023-02-11 09:54:15,224 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=a7bf2dbceab20d06148a05aabdce5a38, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:15,224 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109249956 2023-02-11 09:54:15,224 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109255224"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109255224"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109255224"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109255224"}]},"ts":"1676109255224"} 2023-02-11 09:54:15,225 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109249956 2023-02-11 09:54:15,225 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109250157 2023-02-11 09:54:15,225 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109250157 2023-02-11 09:54:15,225 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109250366 2023-02-11 09:54:15,226 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109250366 2023-02-11 09:54:15,226 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109250574 2023-02-11 09:54:15,226 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109250574 2023-02-11 09:54:15,226 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109250777 2023-02-11 09:54:15,227 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109250777 2023-02-11 09:54:15,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-11 09:54:15,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure a7bf2dbceab20d06148a05aabdce5a38, server=jenkins-hbase4.apache.org,34799,1676109241745 in 380 msec 2023-02-11 09:54:15,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-11 09:54:15,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a7bf2dbceab20d06148a05aabdce5a38, ASSIGN in 542 msec 2023-02-11 09:54:15,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 in 4.1920 sec 2023-02-11 09:54:15,243 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] throttle.PressureAwareThroughputController(145): a7bf2dbceab20d06148a05aabdce5a38#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:15,243 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] throttle.PressureAwareThroughputController(145): a7bf2dbceab20d06148a05aabdce5a38#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:15,261 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/.tmp/cf1/32565181fcab4ef78ca2097b4620cd9b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/32565181fcab4ef78ca2097b4620cd9b 2023-02-11 09:54:15,262 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/.tmp/cf2/9c80da1af79d49429b127d451755e369 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/9c80da1af79d49429b127d451755e369 2023-02-11 09:54:15,268 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a7bf2dbceab20d06148a05aabdce5a38/cf1 of a7bf2dbceab20d06148a05aabdce5a38 into 32565181fcab4ef78ca2097b4620cd9b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:15,268 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a7bf2dbceab20d06148a05aabdce5a38: 2023-02-11 09:54:15,268 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38., storeName=a7bf2dbceab20d06148a05aabdce5a38/cf1, priority=9, startTime=1676109255220; duration=0sec 2023-02-11 09:54:15,268 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:15,269 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a7bf2dbceab20d06148a05aabdce5a38/cf2 of a7bf2dbceab20d06148a05aabdce5a38 into 9c80da1af79d49429b127d451755e369(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:15,269 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a7bf2dbceab20d06148a05aabdce5a38: 2023-02-11 09:54:15,269 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38., storeName=a7bf2dbceab20d06148a05aabdce5a38/cf2, priority=9, startTime=1676109255221; duration=0sec 2023-02-11 09:54:15,269 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:15,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34799] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:38442 deadline: 1676109315433, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8. is not online on jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:15,546 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 09:54:15,554 INFO [Listener at localhost/34597] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 09:54:15,556 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 09:54:15,575 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:15,578 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-11 09:54:15,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-11 09:54:15,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:15,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 09:54:15,583 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109255583"}]},"ts":"1676109255583"} 2023-02-11 09:54:15,585 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-11 09:54:15,587 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-11 09:54:15,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a7bf2dbceab20d06148a05aabdce5a38, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=846a3db876efcaea894eafcf63ddffd7, UNASSIGN}] 2023-02-11 09:54:15,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=846a3db876efcaea894eafcf63ddffd7, UNASSIGN 2023-02-11 09:54:15,589 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=846a3db876efcaea894eafcf63ddffd7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:15,589 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109255589"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109255589"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109255589"}]},"ts":"1676109255589"} 2023-02-11 09:54:15,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 846a3db876efcaea894eafcf63ddffd7, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:15,594 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a7bf2dbceab20d06148a05aabdce5a38, UNASSIGN 2023-02-11 09:54:15,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a7bf2dbceab20d06148a05aabdce5a38, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:15,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109255594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109255594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109255594"}]},"ts":"1676109255594"} 2023-02-11 09:54:15,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure a7bf2dbceab20d06148a05aabdce5a38, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:15,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 09:54:15,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 846a3db876efcaea894eafcf63ddffd7, disabling compactions & flushes 2023-02-11 09:54:15,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. after waiting 0 ms 2023-02-11 09:54:15,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,755 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60-top] to archive 2023-02-11 09:54:15,756 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:15,758 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,759 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,761 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,762 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,763 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,765 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,766 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,768 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b-top] to archive 2023-02-11 09:54:15,769 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:15,770 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,772 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,773 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,774 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,775 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,776 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,777 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:54:15,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:15,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7. 2023-02-11 09:54:15,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 846a3db876efcaea894eafcf63ddffd7: Waiting for close lock at 1676109255743Running coprocessor pre-close hooks at 1676109255743Disabling compacts and flushes for region at 1676109255743Disabling writes for close at 1676109255743Writing region close event to WAL at 1676109255778 (+35 ms)Running coprocessor post-close hooks at 1676109255782 (+4 ms)Closed at 1676109255782 2023-02-11 09:54:15,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a7bf2dbceab20d06148a05aabdce5a38, disabling compactions & flushes 2023-02-11 09:54:15,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. after waiting 0 ms 2023-02-11 09:54:15,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,784 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=846a3db876efcaea894eafcf63ddffd7, regionState=CLOSED 2023-02-11 09:54:15,784 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109255784"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109255784"}]},"ts":"1676109255784"} 2023-02-11 09:54:15,785 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60-bottom] to archive 2023-02-11 09:54:15,786 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:15,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-11 09:54:15,788 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 846a3db876efcaea894eafcf63ddffd7, server=jenkins-hbase4.apache.org,34799,1676109241745 in 195 msec 2023-02-11 09:54:15,790 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=846a3db876efcaea894eafcf63ddffd7, UNASSIGN in 201 msec 2023-02-11 09:54:15,791 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,793 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,794 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,795 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,796 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,798 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b-bottom] to archive 2023-02-11 09:54:15,799 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:15,800 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,801 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,802 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,803 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,804 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,806 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,807 DEBUG [StoreCloser-testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:54:15,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:15,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38. 2023-02-11 09:54:15,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a7bf2dbceab20d06148a05aabdce5a38: Waiting for close lock at 1676109255783Running coprocessor pre-close hooks at 1676109255783Disabling compacts and flushes for region at 1676109255783Disabling writes for close at 1676109255784 (+1 ms)Writing region close event to WAL at 1676109255808 (+24 ms)Running coprocessor post-close hooks at 1676109255811 (+3 ms)Closed at 1676109255811 2023-02-11 09:54:15,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,813 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a7bf2dbceab20d06148a05aabdce5a38, regionState=CLOSED 2023-02-11 09:54:15,813 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109255813"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109255813"}]},"ts":"1676109255813"} 2023-02-11 09:54:15,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-11 09:54:15,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure a7bf2dbceab20d06148a05aabdce5a38, server=jenkins-hbase4.apache.org,34799,1676109241745 in 219 msec 2023-02-11 09:54:15,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-11 09:54:15,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a7bf2dbceab20d06148a05aabdce5a38, UNASSIGN in 229 msec 2023-02-11 09:54:15,820 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109255819"}]},"ts":"1676109255819"} 2023-02-11 09:54:15,821 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-11 09:54:15,823 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-11 09:54:15,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 244 msec 2023-02-11 09:54:15,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 09:54:15,886 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-11 09:54:15,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-11 09:54:15,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:15,889 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:15,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 09:54:15,890 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:15,897 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,897 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,897 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,899 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/recovered.edits] 2023-02-11 09:54:15,899 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/recovered.edits] 2023-02-11 09:54:15,899 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/recovered.edits] 2023-02-11 09:54:15,903 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/.regioninfo to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/.regioninfo 2023-02-11 09:54:15,903 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/.regioninfo to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/.regioninfo 2023-02-11 09:54:15,904 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/1dc77dfaefdc40e6982f661c6e7f324e 2023-02-11 09:54:15,906 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/6498f3473b174284b4a2351c50e3cbf2 2023-02-11 09:54:15,907 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,908 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/711fbfa50895401a8bda4fc448afd8ca 2023-02-11 09:54:15,908 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/1dc77dfaefdc40e6982f661c6e7f324e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,909 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,909 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/72f6a03e93e64072b35d8cc7ceaa4c60 2023-02-11 09:54:15,910 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/6498f3473b174284b4a2351c50e3cbf2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,911 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,911 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/76e8ffb2449b40ee8b8908ecded94db2 2023-02-11 09:54:15,912 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/711fbfa50895401a8bda4fc448afd8ca.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,912 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,913 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/7de12515aebb4f84b9b8b6da454e3af7 2023-02-11 09:54:15,913 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/72f6a03e93e64072b35d8cc7ceaa4c60.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,914 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,914 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf1/fb527345f1f84478b2b9ad4a8ec1991e 2023-02-11 09:54:15,915 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/76e8ffb2449b40ee8b8908ecded94db2.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,915 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,917 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,917 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/7de12515aebb4f84b9b8b6da454e3af7.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,918 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/021f2c69ab274d7484aa46fa75a371e9 2023-02-11 09:54:15,919 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf1/fb527345f1f84478b2b9ad4a8ec1991e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,920 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/05fc5772e02d46c3aa3a967c52f26789 2023-02-11 09:54:15,922 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,922 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/3201634bc3c240e890bf8bb97b13230e 2023-02-11 09:54:15,923 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,924 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/021f2c69ab274d7484aa46fa75a371e9.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,924 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/66666b1be8e8458a951ec423372b7f6b 2023-02-11 09:54:15,925 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,925 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/05fc5772e02d46c3aa3a967c52f26789.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,926 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/6b5c4f9ec56a4e2faef244445e05ed42 2023-02-11 09:54:15,926 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,927 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/3201634bc3c240e890bf8bb97b13230e.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,927 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/a688546bb00a4aa2ab046ef7500bcc3c 2023-02-11 09:54:15,928 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,929 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/66666b1be8e8458a951ec423372b7f6b.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,929 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5 2023-02-11 09:54:15,931 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,931 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/6b5c4f9ec56a4e2faef244445e05ed42.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,933 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/846a3db876efcaea894eafcf63ddffd7/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,933 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/a688546bb00a4aa2ab046ef7500bcc3c.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,933 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/recovered.edits/85.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8/recovered.edits/85.seqid 2023-02-11 09:54:15,934 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,935 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/a7bf2dbceab20d06148a05aabdce5a38/cf2/ad23a8d5dd4b4cae9b542c1f25d35ce5.4b63fe694a11cba94c671f4e6eff38f8 2023-02-11 09:54:15,937 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/2ca29bafd40945bbb1efed9dec906021 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf1/2ca29bafd40945bbb1efed9dec906021 2023-02-11 09:54:15,939 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/32565181fcab4ef78ca2097b4620cd9b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf1/32565181fcab4ef78ca2097b4620cd9b 2023-02-11 09:54:15,940 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/529b25f3141048399d717673926c5545 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/cf2/529b25f3141048399d717673926c5545 2023-02-11 09:54:15,942 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/9c80da1af79d49429b127d451755e369 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/cf2/9c80da1af79d49429b127d451755e369 2023-02-11 09:54:15,948 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7/recovered.edits/90.seqid 2023-02-11 09:54:15,950 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:15,950 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38/recovered.edits/90.seqid 2023-02-11 09:54:15,951 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitWithoutPONR/a7bf2dbceab20d06148a05aabdce5a38 2023-02-11 09:54:15,951 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 09:54:15,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:15,957 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-11 09:54:15,960 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-11 09:54:15,961 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:15,961 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-11 09:54:15,961 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109255961"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:15,961 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109255961"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:15,961 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109255961"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:15,963 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 09:54:15,963 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4b63fe694a11cba94c671f4e6eff38f8, NAME => 'testSplitWithoutPONR,,1676109249055.4b63fe694a11cba94c671f4e6eff38f8.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a7bf2dbceab20d06148a05aabdce5a38, NAME => 'testSplitWithoutPONR,,1676109251041.a7bf2dbceab20d06148a05aabdce5a38.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 846a3db876efcaea894eafcf63ddffd7, NAME => 'testSplitWithoutPONR,41,1676109251041.846a3db876efcaea894eafcf63ddffd7.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 09:54:15,963 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-11 09:54:15,964 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109255964"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:15,965 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-11 09:54:15,968 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:15,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 82 msec 2023-02-11 09:54:15,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 09:54:15,992 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-11 09:54:16,008 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=400 (was 392) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:54906 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:60596 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:34799-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:57770 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:57794 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:56312 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:57808 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:57754 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:57776 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:60566 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:57788 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:56328 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:54576 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:60624 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:57732 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6a75bb5c-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6a75bb5c-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:54646 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1081213750_15 at /127.0.0.1:56298 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 851) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=312 (was 313), ProcessCount=169 (was 169), AvailableMemoryMB=2893 (was 3034) 2023-02-11 09:54:16,023 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=312, ProcessCount=169, AvailableMemoryMB=2892 2023-02-11 09:54:16,023 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:16,026 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:54:16,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:54:16,028 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:16,030 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:54:16,031 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:54:16,032 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,033 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 empty. 2023-02-11 09:54:16,033 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,033 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 09:54:16,044 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:16,046 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6b41352f778cf0f0366b57384c1aaf98, NAME => 'testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:54:16,053 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:16,053 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 6b41352f778cf0f0366b57384c1aaf98, disabling compactions & flushes 2023-02-11 09:54:16,054 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,054 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,054 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. after waiting 0 ms 2023-02-11 09:54:16,054 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,054 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,054 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 6b41352f778cf0f0366b57384c1aaf98: Waiting for close lock at 1676109256053Disabling compacts and flushes for region at 1676109256053Disabling writes for close at 1676109256054 (+1 ms)Writing region close event to WAL at 1676109256054Closed at 1676109256054 2023-02-11 09:54:16,056 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:54:16,057 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109256057"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109256057"}]},"ts":"1676109256057"} 2023-02-11 09:54:16,058 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:54:16,059 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:54:16,059 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109256059"}]},"ts":"1676109256059"} 2023-02-11 09:54:16,061 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 09:54:16,064 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:16,065 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:16,065 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:16,065 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:16,065 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:16,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN}] 2023-02-11 09:54:16,067 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:16,067 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=false 2023-02-11 09:54:16,218 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:16,219 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,219 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109256219"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109256219"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109256219"}]},"ts":"1676109256219"} 2023-02-11 09:54:16,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:16,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b41352f778cf0f0366b57384c1aaf98, NAME => 'testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:16,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:16,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:16,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,377 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,379 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1 2023-02-11 09:54:16,379 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1 2023-02-11 09:54:16,379 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b41352f778cf0f0366b57384c1aaf98 columnFamilyName cf1 2023-02-11 09:54:16,380 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(310): Store=6b41352f778cf0f0366b57384c1aaf98/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:16,380 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,381 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2 2023-02-11 09:54:16,381 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2 2023-02-11 09:54:16,382 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b41352f778cf0f0366b57384c1aaf98 columnFamilyName cf2 2023-02-11 09:54:16,382 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(310): Store=6b41352f778cf0f0366b57384c1aaf98/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:16,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:16,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:16,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:16,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6b41352f778cf0f0366b57384c1aaf98; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64849778, jitterRate=-0.033663004636764526}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:16,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6b41352f778cf0f0366b57384c1aaf98: Running coprocessor pre-open hook at 1676109256376Writing region info on filesystem at 1676109256376Initializing all the Stores at 1676109256377 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109256377Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109256377Cleaning up temporary data from old regions at 1676109256384 (+7 ms)Cleaning up detritus from prior splits at 1676109256384Running coprocessor post-open hooks at 1676109256389 (+5 ms)Region opened successfully at 1676109256389 2023-02-11 09:54:16,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98., pid=43, masterSystemTime=1676109256372 2023-02-11 09:54:16,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,391 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,392 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109256391"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109256391"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109256391"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109256391"}]},"ts":"1676109256391"} 2023-02-11 09:54:16,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-11 09:54:16,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 in 173 msec 2023-02-11 09:54:16,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-11 09:54:16,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN in 330 msec 2023-02-11 09:54:16,398 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:54:16,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109256398"}]},"ts":"1676109256398"} 2023-02-11 09:54:16,399 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 09:54:16,410 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:54:16,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 383 msec 2023-02-11 09:54:16,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:16,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 09:54:16,540 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 09:54:16,540 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,540 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 09:54:16,540 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:54:16,540 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,540 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,541 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,541 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,541 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,541 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,542 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,542 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:16,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,543 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,544 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,544 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,544 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,544 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,544 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,544 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,544 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. started... 2023-02-11 09:54:16,544 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:16,544 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:16,545 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6b41352f778cf0f0366b57384c1aaf98 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,560 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/ca7b7895dc58471fab374b6c32acc681 2023-02-11 09:54:16,576 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 2023-02-11 09:54:16,580 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/ca7b7895dc58471fab374b6c32acc681 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 2023-02-11 09:54:16,585 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:16,586 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 2023-02-11 09:54:16,590 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:16,591 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 46ms, sequenceid=6, compaction requested=false 2023-02-11 09:54:16,591 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-11 09:54:16,592 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:16,592 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,592 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:16,592 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:16,592 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,592 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,592 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,594 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,594 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:16,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:16,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 09:54:16,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f17e48e[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:16,597 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:54:16,597 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:16,599 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:16,599 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:16,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:16,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:16,601 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:16,601 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,601 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:16,602 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:16,602 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,602 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:16,602 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:16,602 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,602 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:16,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:16,602 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:16,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:16,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:16,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:54:16,606 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 09:54:16,606 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:16,606 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:16,606 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 09:54:16,606 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:16,609 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,609 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:16,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,609 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,610 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:16,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,610 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,612 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:16,613 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,613 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,616 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 09:54:16,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:16,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,616 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,617 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,617 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,617 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:16,617 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,617 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,617 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,618 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,618 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,618 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,717 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:16,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:16,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:16,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:16,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 09:54:16,736 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 09:54:16,736 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,736 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 09:54:16,736 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:54:16,737 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,737 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,738 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,738 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,738 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,738 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,739 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,739 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:16,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:16,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,740 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,741 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,742 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,742 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,742 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,742 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:16,742 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:16,742 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,742 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. started... 2023-02-11 09:54:16,742 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6b41352f778cf0f0366b57384c1aaf98 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:16,743 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,745 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,753 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/287b05c9a4a84cb49a0c79e1818e2b28 2023-02-11 09:54:16,768 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/777e02f3a5db482c8fc46626e458ac9e 2023-02-11 09:54:16,773 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/287b05c9a4a84cb49a0c79e1818e2b28 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 2023-02-11 09:54:16,777 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:16,778 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/777e02f3a5db482c8fc46626e458ac9e as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e 2023-02-11 09:54:16,781 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:16,782 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 40ms, sequenceid=19, compaction requested=false 2023-02-11 09:54:16,782 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:16,782 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,782 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:16,782 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:16,783 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,783 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,783 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,785 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,785 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:16,785 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:16,785 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:16,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 09:54:16,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78d11899[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:16,787 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:54:16,787 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,788 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,788 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,788 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,789 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,789 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:16,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:16,789 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:16,794 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:16,794 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:16,794 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:16,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:16,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:16,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:16,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:16,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:16,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:54:16,798 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 09:54:16,798 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:16,798 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:16,798 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 09:54:16,798 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:16,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:16,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,800 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,801 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,801 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,803 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:16,803 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:16,803 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 09:54:16,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,807 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,806 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:16,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,807 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:16,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,806 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:16,807 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:16,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,807 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,808 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,808 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:16,808 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,808 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,808 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:16,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:16,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:16,808 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,808 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,907 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:16,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:16,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:16,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:16,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 09:54:16,940 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 09:54:16,940 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,940 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 09:54:16,940 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:54:16,941 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,941 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,943 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,943 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,943 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,943 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:16,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:16,944 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:16,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:16,944 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:16,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,945 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:16,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:16,945 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:16,946 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:16,946 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. started... 2023-02-11 09:54:16,946 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6b41352f778cf0f0366b57384c1aaf98 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:16,947 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,948 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,948 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,948 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,948 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:16,949 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:16,969 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/c0588d97e5204bbe858b4c8132c8bcc3 2023-02-11 09:54:16,991 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/5100dc2f042447e4a827b4ecae082890 2023-02-11 09:54:17,002 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/c0588d97e5204bbe858b4c8132c8bcc3 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 2023-02-11 09:54:17,008 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:17,009 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/5100dc2f042447e4a827b4ecae082890 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 2023-02-11 09:54:17,014 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:17,015 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 69ms, sequenceid=32, compaction requested=true 2023-02-11 09:54:17,016 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:17,016 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:17,016 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:17,016 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:17,016 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,016 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,016 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,018 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,018 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,018 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,019 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 09:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6869c323[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:17,022 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:54:17,022 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,025 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,025 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,025 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,026 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,026 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,026 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:17,026 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,027 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:17,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,028 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,028 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,028 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:17,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,032 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,032 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,032 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,034 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,034 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,035 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:17,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:54:17,039 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 09:54:17,039 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:17,039 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:17,039 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 09:54:17,039 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:17,041 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,041 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,041 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,041 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,041 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,041 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,042 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,043 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,044 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,044 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,046 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,047 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,047 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 09:54:17,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,055 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:17,055 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:17,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,057 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,156 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:17,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:17,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:17,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:17,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 09:54:17,184 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 09:54:17,184 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,184 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 09:54:17,185 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:54:17,185 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,185 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,187 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,187 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,187 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,187 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,188 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,188 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,192 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,192 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,192 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,192 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,209 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,209 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,209 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,209 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,209 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,209 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,209 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,209 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,210 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,210 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,210 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,210 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,214 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:17,214 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:17,214 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:17,214 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. started... 2023-02-11 09:54:17,214 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6b41352f778cf0f0366b57384c1aaf98 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:17,233 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/1e76032f2ff44818b568481b7b724314 2023-02-11 09:54:17,249 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/1952b34a1e4a4c0d9661f72bdbe082bc 2023-02-11 09:54:17,254 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/1e76032f2ff44818b568481b7b724314 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 2023-02-11 09:54:17,259 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:17,260 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/1952b34a1e4a4c0d9661f72bdbe082bc as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc 2023-02-11 09:54:17,264 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:17,265 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 51ms, sequenceid=45, compaction requested=true 2023-02-11 09:54:17,265 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:17,265 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:17,265 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:17,265 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:17,266 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,266 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,266 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,267 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,267 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,268 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,268 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 09:54:17,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67db6606[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:17,270 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:54:17,270 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,273 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:17,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,274 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:17,274 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:17,275 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,275 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,275 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,275 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,276 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,276 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,276 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,276 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,276 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,276 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:17,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:54:17,279 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 09:54:17,279 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:17,279 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:17,279 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 09:54:17,279 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:17,281 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,281 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,282 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,282 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,282 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,283 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,284 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,285 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,285 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,291 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,291 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,291 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,291 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,291 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 09:54:17,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:17,291 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,292 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,292 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,392 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:17,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:17,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:17,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:17,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 09:54:17,411 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 09:54:17,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 09:54:17,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:54:17,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,413 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,413 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,414 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:17,415 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:17,415 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. started... 2023-02-11 09:54:17,416 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6b41352f778cf0f0366b57384c1aaf98 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,431 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 2023-02-11 09:54:17,443 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/ebc6eaace0154020b94677c646c9c727 2023-02-11 09:54:17,448 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 2023-02-11 09:54:17,451 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:17,452 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/ebc6eaace0154020b94677c646c9c727 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 2023-02-11 09:54:17,455 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:17,456 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 41ms, sequenceid=58, compaction requested=true 2023-02-11 09:54:17,456 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:17,456 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:17,456 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:17,456 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:17,456 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,456 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,456 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,458 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,458 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,458 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,458 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 09:54:17,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1798dde7[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:17,460 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:54:17,460 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,462 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,462 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,462 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,463 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,463 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:17,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:17,463 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:17,464 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,464 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,464 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,468 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,468 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,468 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,468 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,468 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,468 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:17,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:54:17,473 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 09:54:17,473 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:17,473 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:17,473 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 09:54:17,473 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:17,476 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,476 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,477 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,477 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,477 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,478 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,479 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,480 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,480 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 09:54:17,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:17,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,483 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,484 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,484 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:17,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,584 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:17,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:17,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:17,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:17,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 09:54:17,601 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 09:54:17,601 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,601 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 09:54:17,601 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:54:17,602 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,602 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,603 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,603 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,603 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,603 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,603 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,603 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:17,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 09:54:17,605 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,605 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,605 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:17,605 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,605 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:17,606 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. started... 2023-02-11 09:54:17,606 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6b41352f778cf0f0366b57384c1aaf98 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:17,607 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,609 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,609 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,609 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,609 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,609 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,615 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/077d1d25deda4b23b3dc948d41b6132a 2023-02-11 09:54:17,629 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/b6da5f6d276c4e74929a7f1b81a76b42 2023-02-11 09:54:17,635 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/077d1d25deda4b23b3dc948d41b6132a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a 2023-02-11 09:54:17,640 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:17,641 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/b6da5f6d276c4e74929a7f1b81a76b42 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 2023-02-11 09:54:17,645 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:17,646 INFO [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 40ms, sequenceid=71, compaction requested=true 2023-02-11 09:54:17,646 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:17,646 DEBUG [rs(jenkins-hbase4.apache.org,34799,1676109241745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:17,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:17,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:17,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 09:54:17,647 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 09:54:17,648 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,648 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,649 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,649 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:17,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 09:54:17,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67fd0a5d[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:17,651 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:54:17,651 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,652 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,652 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:17,652 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,652 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:17,652 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 09:54:17,656 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 09:54:17,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34799,1676109241745': 2023-02-11 09:54:17,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:54:17,659 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 09:54:17,660 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:17,660 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:17,660 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 09:54:17,660 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:17,661 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,661 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:17,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:17,661 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:17,662 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,663 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,663 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:17,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,670 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,671 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:17,671 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 09:54:17,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,681 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:17,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 09:54:17,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:17,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 09:54:17,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:17,681 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:17,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:17,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:17,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:17,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:17,682 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 09:54:17,683 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 09:54:17,782 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:17,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:18,043 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 09:54:18,044 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:18,045 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,046 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,046 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,046 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,047 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:18,047 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:18,047 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:18,295 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,296 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:18,296 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:18,296 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:18,296 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-11 09:54:18,296 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:18,296 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:18,296 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:18,296 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:18,297 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:18,298 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 09:54:18,298 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:18,299 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:18,299 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:18,299 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:18,299 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,300 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:18,300 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,300 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,300 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,300 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:18,301 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,301 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,301 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:18,302 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,302 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:18,302 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,302 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,302 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,302 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:18,303 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:18,304 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:18,304 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109288304 2023-02-11 09:54:18,304 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:18,306 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,308 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:18,309 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:18,309 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:18,309 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109318309 2023-02-11 09:54:18,309 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109378309 2023-02-11 09:54:18,309 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 09:54:18,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:18,313 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:18,313 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:18,313 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:18,559 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,560 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:18,560 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:18,560 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:18,561 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:18,561 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:18,561 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:18,561 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:18,561 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:18,562 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:18,563 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 09:54:18,563 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:18,563 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:18,563 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:18,563 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:18,564 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,564 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:18,564 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,564 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,565 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,565 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:18,565 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,565 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,566 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:18,566 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,566 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:18,566 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,566 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,566 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,566 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 09:54:18,567 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:18,567 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:18,568 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109288568 2023-02-11 09:54:18,568 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:18,569 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,571 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:18,571 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:18,571 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:18,571 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109318571 2023-02-11 09:54:18,571 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109378571 2023-02-11 09:54:18,572 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:18,572 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:18,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN}] 2023-02-11 09:54:18,574 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:18,574 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:18,574 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:18,822 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,822 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:18,822 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:18,822 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:18,824 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:18,824 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:18,824 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:18,824 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:18,825 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:18,825 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:18,826 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 09:54:18,827 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:18,827 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:18,827 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:18,827 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:18,827 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,828 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:18,828 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,828 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:18,828 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,829 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:18,829 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,829 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:18,830 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:18,830 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,830 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:18,830 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,830 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,830 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,830 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 09:54:18,831 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:18,831 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:18,832 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109288832 2023-02-11 09:54:18,832 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:18,833 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:18,835 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:18,835 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:18,835 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:18,836 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109318836 2023-02-11 09:54:18,836 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109378836 2023-02-11 09:54:18,836 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:18,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN}] 2023-02-11 09:54:18,836 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:18,838 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:18,839 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:18,839 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109258839"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109258839"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109258839"}]},"ts":"1676109258839"} 2023-02-11 09:54:18,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:18,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:18,841 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:18,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:18,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:19,086 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,086 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:19,086 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:19,086 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:19,086 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:19,086 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:19,087 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:19,087 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:19,087 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:19,087 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:19,088 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:19,088 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 09:54:19,089 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:19,089 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:19,089 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:19,089 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:19,090 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,090 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:19,090 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,090 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,090 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,091 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:19,091 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,091 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,091 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:19,092 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,092 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,092 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:19,092 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,092 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,092 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,092 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:19,093 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,093 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:19,093 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:19,094 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:19,094 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:19,094 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109289094 2023-02-11 09:54:19,095 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:19,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,097 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,097 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,097 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:19,097 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109319097 2023-02-11 09:54:19,097 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109379097 2023-02-11 09:54:19,097 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:54:19,098 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:19,098 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109259098"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109259098"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109259098"}]},"ts":"1676109259098"} 2023-02-11 09:54:19,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:19,099 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:19,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6b41352f778cf0f0366b57384c1aaf98, disabling compactions & flushes 2023-02-11 09:54:19,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:19,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:19,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. after waiting 0 ms 2023-02-11 09:54:19,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:19,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 6b41352f778cf0f0366b57384c1aaf98 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 09:54:19,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/6fa7cbab739543ef9b4dfe833fec07e7 2023-02-11 09:54:19,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/f6951c34a7364d7281aaabd4812502dd 2023-02-11 09:54:19,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/6fa7cbab739543ef9b4dfe833fec07e7 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 2023-02-11 09:54:19,293 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:54:19,294 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/f6951c34a7364d7281aaabd4812502dd as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd 2023-02-11 09:54:19,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:54:19,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 47ms, sequenceid=82, compaction requested=true 2023-02-11 09:54:19,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 09:54:19,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:19,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:19,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6b41352f778cf0f0366b57384c1aaf98: Waiting for close lock at 1676109259252Running coprocessor pre-close hooks at 1676109259252Disabling compacts and flushes for region at 1676109259252Disabling writes for close at 1676109259252Obtaining lock to block concurrent updates at 1676109259252Preparing flush snapshotting stores in 6b41352f778cf0f0366b57384c1aaf98 at 1676109259252Finished memstore snapshotting testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676109259253 (+1 ms)Flushing stores of testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. at 1676109259253Flushing 6b41352f778cf0f0366b57384c1aaf98/cf1: creating writer at 1676109259254 (+1 ms)Flushing 6b41352f778cf0f0366b57384c1aaf98/cf1: appending metadata at 1676109259256 (+2 ms)Flushing 6b41352f778cf0f0366b57384c1aaf98/cf1: closing flushed file at 1676109259256Flushing 6b41352f778cf0f0366b57384c1aaf98/cf2: creating writer at 1676109259268 (+12 ms)Flushing 6b41352f778cf0f0366b57384c1aaf98/cf2: appending metadata at 1676109259270 (+2 ms)Flushing 6b41352f778cf0f0366b57384c1aaf98/cf2: closing flushed file at 1676109259270Flushing 6b41352f778cf0f0366b57384c1aaf98/cf1: reopening flushed file at 1676109259289 (+19 ms)Flushing 6b41352f778cf0f0366b57384c1aaf98/cf2: reopening flushed file at 1676109259294 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6b41352f778cf0f0366b57384c1aaf98 in 47ms, sequenceid=82, compaction requested=true at 1676109259299 (+5 ms)Writing region close event to WAL at 1676109259301 (+2 ms)Running coprocessor post-close hooks at 1676109259306 (+5 ms)Closed at 1676109259306 2023-02-11 09:54:19,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,309 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED 2023-02-11 09:54:19,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109259309"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109259309"}]},"ts":"1676109259309"} 2023-02-11 09:54:19,311 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:19,311 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:19,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:19,311 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:19,348 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:19,348 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:19,349 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:19,350 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:19,350 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 09:54:19,350 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 09:54:19,351 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:19,351 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:19,352 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:19,352 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:19,352 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,353 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:19,353 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,353 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,354 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,354 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:19,354 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,355 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,355 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:19,356 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,356 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,356 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:19,356 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:19,356 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,356 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,356 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,357 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:19,357 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,357 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:19,357 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 09:54:19,358 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:19,358 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:19,358 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:19,361 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109289361 2023-02-11 09:54:19,362 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:19,362 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:19,365 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,365 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,366 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:19,366 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109319366 2023-02-11 09:54:19,366 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109379366 2023-02-11 09:54:19,366 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:19,366 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:19,366 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109259366"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109259366"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109259366"}]},"ts":"1676109259366"} 2023-02-11 09:54:19,367 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:19,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-11 09:54:19,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 in 268 msec 2023-02-11 09:54:19,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:19,371 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:19,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:19,616 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,616 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:19,616 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:19,616 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:19,617 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:19,617 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:19,617 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:19,617 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:19,617 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:19,618 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:19,619 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:19,619 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 09:54:19,619 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:19,620 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:19,621 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:19,621 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:19,621 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:19,621 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,622 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:19,622 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,622 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,623 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,623 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:19,623 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,623 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,624 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:19,625 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,625 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,625 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:19,625 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:19,625 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,625 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,625 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,625 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:19,625 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:19,625 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:54:19,626 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:19,626 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:19,626 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:19,627 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109289627 2023-02-11 09:54:19,627 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:19,628 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,630 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,630 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,630 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:19,630 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109319630 2023-02-11 09:54:19,630 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109379630 2023-02-11 09:54:19,630 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:19,630 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:19,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-11 09:54:19,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN in 796 msec 2023-02-11 09:54:19,637 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:19,637 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:19,637 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:19,880 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,880 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:19,880 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:19,881 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:19,882 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:19,882 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:19,882 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:19,882 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:19,882 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:19,883 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:19,884 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:19,884 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:19,884 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:19,884 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:19,885 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:19,885 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:19,885 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:19,885 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,886 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:19,886 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,886 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:19,886 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,887 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:19,887 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,887 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:19,887 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:19,888 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,888 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:19,888 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:19,888 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:19,888 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:19,888 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:19,888 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:19,891 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:19,892 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:19,898 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109289898 2023-02-11 09:54:19,899 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,899 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:19,902 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,902 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:19,902 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:19,902 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109319902 2023-02-11 09:54:19,902 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109379902 2023-02-11 09:54:19,902 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:54:19,902 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,902 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,902 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:19,907 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:19,916 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=6b41352f778cf0f0366b57384c1aaf98, threads=14 2023-02-11 09:54:19,917 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,917 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,917 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,917 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,917 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,917 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,917 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,918 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,918 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,918 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,918 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,918 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,919 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:19,919 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,012 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,026 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,027 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,033 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,050 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,052 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,052 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,056 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,056 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,060 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,063 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,068 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,068 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,454 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,455 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 6b41352f778cf0f0366b57384c1aaf98 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:54:20,505 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:54:20,898 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:20,899 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:20,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:20,899 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:20,903 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:20,903 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:20,904 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:20,905 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 09:54:20,905 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:20,905 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:20,905 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:20,906 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:20,906 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:20,906 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:20,906 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:20,906 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:20,907 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:20,907 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:20,907 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:20,907 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:20,907 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:20,907 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:20,908 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:20,908 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:20,908 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:20,908 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:20,908 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:20,909 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:20,909 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:20,909 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:20,909 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:20,909 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:20,909 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 09:54:20,909 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:20,910 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:20,914 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109290914 2023-02-11 09:54:20,916 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:20,916 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:20,919 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:20,919 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:20,919 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:20,919 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109320919 2023-02-11 09:54:20,919 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109380919 2023-02-11 09:54:20,919 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:20,929 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=6b41352f778cf0f0366b57384c1aaf98, threads=14 2023-02-11 09:54:20,930 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,930 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,930 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,930 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,930 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,930 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,930 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,931 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,931 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,931 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,931 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,931 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,931 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:20,931 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,034 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,058 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,067 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,073 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,098 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,098 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,108 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,116 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,119 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,120 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,120 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,120 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,122 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,122 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc for region: 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:21,122 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 6b41352f778cf0f0366b57384c1aaf98 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:54:21,163 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:21,169 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/0aacff6a4012145ca0279099292e93be/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:54:21,170 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/87f85251e995996fd01900699cf6571c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:54:21,171 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:21,171 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:21,171 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:21,420 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,420 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:21,420 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:21,420 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:21,421 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:21,421 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:21,421 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:21,421 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:21,421 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:21,422 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:21,423 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 09:54:21,423 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:21,423 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:21,423 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:21,424 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:21,424 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:21,424 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:21,425 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,425 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:21,426 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,426 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,426 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,427 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:21,427 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,427 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,427 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:21,428 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,428 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:21,428 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:21,430 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:21,430 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,430 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,431 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,431 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:21,431 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:21,431 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-11 09:54:21,431 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:21,432 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:21,432 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109291432 2023-02-11 09:54:21,433 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:21,433 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,436 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:21,436 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:21,436 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:21,437 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109321437 2023-02-11 09:54:21,437 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109381437 2023-02-11 09:54:21,437 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:21,440 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:21,443 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:21,443 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:21,443 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:21,688 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,688 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:21,688 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:21,688 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:21,689 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:21,689 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:21,689 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:21,689 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:21,690 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:21,690 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:21,692 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 09:54:21,692 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:21,692 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:21,692 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:21,693 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:21,693 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:21,693 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:21,694 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,694 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:21,694 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,694 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,695 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,695 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:21,695 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,695 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,696 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:21,696 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,697 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:21,697 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:21,697 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:21,697 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,697 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,697 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,697 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:21,697 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:21,697 DEBUG [Listener at localhost/34597] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,697 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 09:54:21,699 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:21,699 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:21,707 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109291707 2023-02-11 09:54:21,708 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:21,709 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,712 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:21,712 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:21,712 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:21,712 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109321712 2023-02-11 09:54:21,713 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109381713 2023-02-11 09:54:21,713 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 09:54:21,713 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:21,727 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:21,727 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 09:54:21,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:21,727 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:21,964 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,964 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:21,964 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:21,964 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:21,964 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:21,964 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:21,965 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:21,965 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:21,965 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:21,965 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:21,966 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 09:54:21,967 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:21,967 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:21,967 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:21,967 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:21,967 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:21,968 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:21,968 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,968 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:21,968 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,968 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:21,969 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,969 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:21,969 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,969 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:21,969 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:21,970 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,970 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:21,970 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:21,970 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:21,970 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,970 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,970 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,970 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:21,971 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:21,971 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:21,971 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:21,972 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:21,972 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109291972 2023-02-11 09:54:21,974 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:21,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:21,977 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:21,977 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:21,977 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:21,977 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109321977 2023-02-11 09:54:21,977 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109381977 2023-02-11 09:54:21,977 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:21,978 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:21,980 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:21,980 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 09:54:21,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:21,980 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:22,228 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,229 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:22,229 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:22,229 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:22,229 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:22,230 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:22,230 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:22,230 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:22,230 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:22,230 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:22,231 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 09:54:22,232 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:22,232 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:22,232 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:22,232 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:22,232 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:22,232 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:22,233 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,233 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:22,233 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,233 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,234 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,234 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:22,234 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,234 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,234 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:22,235 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,235 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:22,235 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:22,235 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:22,235 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,235 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,236 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,236 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:22,236 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:22,236 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:22,236 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:22,237 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:22,237 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109292237 2023-02-11 09:54:22,238 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:22,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,241 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:22,241 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:22,241 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:22,241 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109322241 2023-02-11 09:54:22,241 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109382241 2023-02-11 09:54:22,241 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:22,241 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:22,243 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:22,243 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 09:54:22,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:22,243 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:22,491 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,492 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:22,492 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:22,492 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:22,493 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:22,493 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:22,493 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:22,493 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:22,493 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:22,494 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:22,494 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 09:54:22,495 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:22,495 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:22,495 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:22,495 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:22,495 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:22,495 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:22,496 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,496 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:22,496 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,496 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,497 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,497 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:22,497 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,497 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,497 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:22,498 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,498 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:22,498 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:22,498 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:22,498 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,498 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,498 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,498 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:22,498 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:22,498 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 09:54:22,499 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:22,500 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:22,500 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109292500 2023-02-11 09:54:22,501 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:22,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,503 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:22,504 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:22,504 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:22,504 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109322504 2023-02-11 09:54:22,504 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109382504 2023-02-11 09:54:22,504 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:54:22,504 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:22,507 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN] 2023-02-11 09:54:22,507 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:22,508 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 09:54:22,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:22,508 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:22,754 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,754 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:22,754 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:22,754 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:22,755 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:22,755 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:22,755 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:22,755 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:22,755 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:22,756 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:22,756 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 09:54:22,757 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:54:22,757 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:54:22,757 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 09:54:22,757 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:22,757 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:22,757 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:22,758 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:22,758 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,758 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:22,758 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,758 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:22,758 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,759 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:22,759 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,759 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:22,759 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:22,760 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,760 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:22,760 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:22,760 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:22,760 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:22,760 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,761 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,761 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,761 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:22,761 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:22,761 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:22,761 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:22,761 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:22,762 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:22,762 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:22,762 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109292762 2023-02-11 09:54:22,762 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:22,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:22,765 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:22,767 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:22,767 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:22,767 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:22,767 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109322767 2023-02-11 09:54:22,767 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109382767 2023-02-11 09:54:22,767 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 09:54:22,768 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:22,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:22,773 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 09:54:22,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:22,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:23,018 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,018 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:23,018 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:23,018 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:23,019 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:23,019 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:23,019 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:23,019 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:23,019 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:23,020 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:23,021 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:54:23,021 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 09:54:23,021 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:23,021 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:23,021 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:23,022 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:23,022 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,022 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:23,022 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,022 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,023 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,023 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:23,023 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,023 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,023 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:23,024 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,024 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,024 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:23,024 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,024 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,024 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,024 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:23,024 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 09:54:23,025 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:23,025 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:23,025 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:23,026 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109293026 2023-02-11 09:54:23,026 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:23,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,028 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,030 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,030 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,030 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:23,030 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109323030 2023-02-11 09:54:23,031 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109383031 2023-02-11 09:54:23,031 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 09:54:23,031 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:23,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:23,033 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 09:54:23,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:23,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:23,281 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,281 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:23,281 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:23,281 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:23,283 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:23,283 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:23,284 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:23,284 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:23,284 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:23,284 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:23,285 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 09:54:23,285 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 09:54:23,285 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:23,286 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:23,286 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:23,286 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:23,286 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,286 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:23,287 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,287 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,287 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,288 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:23,288 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,288 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,288 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:23,289 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,289 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,289 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:23,289 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,289 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,289 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,290 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:23,290 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 09:54:23,290 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:23,291 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:23,291 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:23,291 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109293291 2023-02-11 09:54:23,292 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:23,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,293 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,295 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,295 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,295 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:23,295 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109323295 2023-02-11 09:54:23,296 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109383296 2023-02-11 09:54:23,296 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:54:23,296 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:23,298 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:23,298 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 09:54:23,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:23,298 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:23,546 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,546 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:23,546 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:23,546 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:23,546 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:23,546 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:23,546 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:23,546 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:23,547 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:23,547 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:23,548 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 09:54:23,548 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 09:54:23,548 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:23,549 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:23,549 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:23,549 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:23,549 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,549 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:23,549 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,550 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,550 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,550 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:23,550 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,550 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,550 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:23,551 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,551 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,551 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:23,551 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,551 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,551 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,551 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:54:23,551 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-11 09:54:23,552 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:23,552 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:23,552 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:23,553 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109293553 2023-02-11 09:54:23,553 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:23,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,557 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,559 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,559 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,559 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:23,559 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109323559 2023-02-11 09:54:23,559 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109383559 2023-02-11 09:54:23,559 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-11 09:54:23,560 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:54:23,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c exec-time=5.5160 sec 2023-02-11 09:54:23,562 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,563 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=true 2023-02-11 09:54:23,713 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:23,714 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:23,714 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109263714"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109263714"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109263714"}]},"ts":"1676109263714"} 2023-02-11 09:54:23,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:23,716 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:54:23,716 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:54:23,716 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:54:23,810 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:23,810 WARN [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-11 09:54:23,810 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:54:23,810 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:54:23,810 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:54:23,810 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:54:23,810 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:54:23,810 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:54:23,810 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:54:23,810 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:54:23,811 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:54:23,812 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-11 09:54:23,812 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:54:23,813 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:54:23,813 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:54:23,813 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:54:23,813 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,813 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:54:23,813 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,814 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:54:23,814 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,814 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:54:23,814 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,814 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:54:23,815 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:23,815 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:54:23,815 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,815 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:54:23,815 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,815 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-11 09:54:23,816 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:54:23,816 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:54:23,816 INFO [Listener at localhost/34597] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98 to restore RIT 2023-02-11 09:54:23,817 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109293817 2023-02-11 09:54:23,818 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:54:23,818 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:54:23,819 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,820 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=6b41352f778cf0f0366b57384c1aaf98, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:54:23,820 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:54:23,820 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109323820 2023-02-11 09:54:23,820 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109383820 2023-02-11 09:54:23,820 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:54:23,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:23,820 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109263820"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109263820"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109263820"}]},"ts":"1676109263820"} 2023-02-11 09:54:23,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:23,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:23,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b41352f778cf0f0366b57384c1aaf98, NAME => 'testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:23,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:23,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:23,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:23,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:23,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:23,978 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:23,979 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1 2023-02-11 09:54:23,979 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1 2023-02-11 09:54:23,980 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b41352f778cf0f0366b57384c1aaf98 columnFamilyName cf1 2023-02-11 09:54:23,986 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a 2023-02-11 09:54:23,989 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 2023-02-11 09:54:23,992 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 2023-02-11 09:54:23,995 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 2023-02-11 09:54:23,998 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 2023-02-11 09:54:24,000 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 2023-02-11 09:54:24,003 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 2023-02-11 09:54:24,003 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(310): Store=6b41352f778cf0f0366b57384c1aaf98/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:24,004 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,004 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2 2023-02-11 09:54:24,004 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2 2023-02-11 09:54:24,005 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b41352f778cf0f0366b57384c1aaf98 columnFamilyName cf2 2023-02-11 09:54:24,010 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc 2023-02-11 09:54:24,015 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 2023-02-11 09:54:24,018 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e 2023-02-11 09:54:24,020 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 2023-02-11 09:54:24,024 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 2023-02-11 09:54:24,027 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 2023-02-11 09:54:24,030 DEBUG [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd 2023-02-11 09:54:24,030 INFO [StoreOpener-6b41352f778cf0f0366b57384c1aaf98-1] regionserver.HStore(310): Store=6b41352f778cf0f0366b57384c1aaf98/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:24,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.splits 2023-02-11 09:54:24,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:24,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6b41352f778cf0f0366b57384c1aaf98; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72549244, jitterRate=0.08106797933578491}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:24,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6b41352f778cf0f0366b57384c1aaf98: Running coprocessor pre-open hook at 1676109263977Writing region info on filesystem at 1676109263977Initializing all the Stores at 1676109263978 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109263978Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109263978Cleaning up temporary data from old regions at 1676109264033 (+55 ms)Cleaning up detritus from prior splits at 1676109264033Running coprocessor post-open hooks at 1676109264036 (+3 ms)Region opened successfully at 1676109264036 2023-02-11 09:54:24,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98., pid=48, masterSystemTime=1676109263974 2023-02-11 09:54:24,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:24,038 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:24,038 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:24,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:24,039 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 09:54:24,039 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 09:54:24,039 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1912): 6b41352f778cf0f0366b57384c1aaf98/cf2 is initiating minor compaction (all files) 2023-02-11 09:54:24,039 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1912): 6b41352f778cf0f0366b57384c1aaf98/cf1 is initiating minor compaction (all files) 2023-02-11 09:54:24,039 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b41352f778cf0f0366b57384c1aaf98/cf2 in testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,039 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b41352f778cf0f0366b57384c1aaf98/cf1 in testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,039 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp, totalSize=34.9 K 2023-02-11 09:54:24,039 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp, totalSize=34.9 K 2023-02-11 09:54:24,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,040 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting ca7b7895dc58471fab374b6c32acc681, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109256531 2023-02-11 09:54:24,040 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 88c42c40e7a54a0d9b68ce932c2e92c0, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109256531 2023-02-11 09:54:24,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,041 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 287b05c9a4a84cb49a0c79e1818e2b28, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109256719 2023-02-11 09:54:24,041 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:24,041 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109264041"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109264041"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109264041"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109264041"}]},"ts":"1676109264041"} 2023-02-11 09:54:24,041 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 777e02f3a5db482c8fc46626e458ac9e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109256719 2023-02-11 09:54:24,041 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting c0588d97e5204bbe858b4c8132c8bcc3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109256909 2023-02-11 09:54:24,041 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 5100dc2f042447e4a827b4ecae082890, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109256909 2023-02-11 09:54:24,042 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 1e76032f2ff44818b568481b7b724314, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109257159 2023-02-11 09:54:24,042 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting 1952b34a1e4a4c0d9661f72bdbe082bc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109257159 2023-02-11 09:54:24,042 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 3e65c7e9f50f4d398cc9e51e81be19b0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109257394 2023-02-11 09:54:24,042 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting ebc6eaace0154020b94677c646c9c727, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109257394 2023-02-11 09:54:24,042 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 077d1d25deda4b23b3dc948d41b6132a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109257586 2023-02-11 09:54:24,043 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting b6da5f6d276c4e74929a7f1b81a76b42, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109257586 2023-02-11 09:54:24,043 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] compactions.Compactor(207): Compacting 6fa7cbab739543ef9b4dfe833fec07e7, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109257784 2023-02-11 09:54:24,043 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] compactions.Compactor(207): Compacting f6951c34a7364d7281aaabd4812502dd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109257784 2023-02-11 09:54:24,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-11 09:54:24,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 in 223 msec 2023-02-11 09:54:24,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN in 1.5400 sec 2023-02-11 09:54:24,068 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6b41352f778cf0f0366b57384c1aaf98#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:24,068 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] throttle.PressureAwareThroughputController(145): 6b41352f778cf0f0366b57384c1aaf98#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:24,086 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf1/00f2f346a45842ad81affe72b28a5e90 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/00f2f346a45842ad81affe72b28a5e90 2023-02-11 09:54:24,091 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b41352f778cf0f0366b57384c1aaf98/cf1 of 6b41352f778cf0f0366b57384c1aaf98 into 00f2f346a45842ad81affe72b28a5e90(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:24,091 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:24,091 INFO [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98., storeName=6b41352f778cf0f0366b57384c1aaf98/cf1, priority=9, startTime=1676109264037; duration=0sec 2023-02-11 09:54:24,095 DEBUG [RS:1;jenkins-hbase4:34799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:24,101 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/.tmp/cf2/2d6eef0f4eb948708c5e00e35a581317 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/2d6eef0f4eb948708c5e00e35a581317 2023-02-11 09:54:24,106 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b41352f778cf0f0366b57384c1aaf98/cf2 of 6b41352f778cf0f0366b57384c1aaf98 into 2d6eef0f4eb948708c5e00e35a581317(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:24,106 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b41352f778cf0f0366b57384c1aaf98: 2023-02-11 09:54:24,106 INFO [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98., storeName=6b41352f778cf0f0366b57384c1aaf98/cf2, priority=9, startTime=1676109264038; duration=0sec 2023-02-11 09:54:24,106 DEBUG [RS:1;jenkins-hbase4:34799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:24,397 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-11 09:54:24,399 DEBUG [Listener at localhost/34597] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-11 09:54:24,401 INFO [Listener at localhost/34597] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 09:54:24,403 INFO [Listener at localhost/34597] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-11 09:54:24,404 INFO [Listener at localhost/34597] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 09:54:24,404 INFO [Listener at localhost/34597] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-11 09:54:24,420 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:24,421 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-11 09:54:24,422 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-11 09:54:24,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:24,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 09:54:24,425 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109264425"}]},"ts":"1676109264425"} 2023-02-11 09:54:24,426 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 09:54:24,428 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-11 09:54:24,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN}] 2023-02-11 09:54:24,429 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN 2023-02-11 09:54:24,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:24,430 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109264430"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109264430"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109264430"}]},"ts":"1676109264430"} 2023-02-11 09:54:24,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:24,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 09:54:24,583 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6b41352f778cf0f0366b57384c1aaf98, disabling compactions & flushes 2023-02-11 09:54:24,583 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. after waiting 0 ms 2023-02-11 09:54:24,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,584 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7] to archive 2023-02-11 09:54:24,585 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:24,586 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/ca7b7895dc58471fab374b6c32acc681 2023-02-11 09:54:24,588 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/287b05c9a4a84cb49a0c79e1818e2b28 2023-02-11 09:54:24,589 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/c0588d97e5204bbe858b4c8132c8bcc3 2023-02-11 09:54:24,590 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/1e76032f2ff44818b568481b7b724314 2023-02-11 09:54:24,591 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/3e65c7e9f50f4d398cc9e51e81be19b0 2023-02-11 09:54:24,593 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/077d1d25deda4b23b3dc948d41b6132a 2023-02-11 09:54:24,594 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/6fa7cbab739543ef9b4dfe833fec07e7 2023-02-11 09:54:24,598 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd] to archive 2023-02-11 09:54:24,599 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:24,600 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/88c42c40e7a54a0d9b68ce932c2e92c0 2023-02-11 09:54:24,601 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/777e02f3a5db482c8fc46626e458ac9e 2023-02-11 09:54:24,602 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/5100dc2f042447e4a827b4ecae082890 2023-02-11 09:54:24,603 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/1952b34a1e4a4c0d9661f72bdbe082bc 2023-02-11 09:54:24,604 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/ebc6eaace0154020b94677c646c9c727 2023-02-11 09:54:24,605 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/b6da5f6d276c4e74929a7f1b81a76b42 2023-02-11 09:54:24,606 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/f6951c34a7364d7281aaabd4812502dd 2023-02-11 09:54:24,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:54:24,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:24,610 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98. 2023-02-11 09:54:24,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6b41352f778cf0f0366b57384c1aaf98: Waiting for close lock at 1676109264583Running coprocessor pre-close hooks at 1676109264583Disabling compacts and flushes for region at 1676109264583Disabling writes for close at 1676109264583Writing region close event to WAL at 1676109264607 (+24 ms)Running coprocessor post-close hooks at 1676109264610 (+3 ms)Closed at 1676109264610 2023-02-11 09:54:24,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,612 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=6b41352f778cf0f0366b57384c1aaf98, regionState=CLOSED 2023-02-11 09:54:24,612 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109264612"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109264612"}]},"ts":"1676109264612"} 2023-02-11 09:54:24,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-11 09:54:24,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 6b41352f778cf0f0366b57384c1aaf98, server=jenkins-hbase4.apache.org,34799,1676109241745 in 182 msec 2023-02-11 09:54:24,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-11 09:54:24,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, UNASSIGN in 186 msec 2023-02-11 09:54:24,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109264617"}]},"ts":"1676109264617"} 2023-02-11 09:54:24,618 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 09:54:24,620 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-11 09:54:24,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 198 msec 2023-02-11 09:54:24,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 09:54:24,727 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-11 09:54:24,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-11 09:54:24,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:24,731 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:24,731 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:24,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 09:54:24,735 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,737 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/recovered.edits] 2023-02-11 09:54:24,740 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/00f2f346a45842ad81affe72b28a5e90 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf1/00f2f346a45842ad81affe72b28a5e90 2023-02-11 09:54:24,742 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/2d6eef0f4eb948708c5e00e35a581317 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/cf2/2d6eef0f4eb948708c5e00e35a581317 2023-02-11 09:54:24,745 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98/recovered.edits/90.seqid 2023-02-11 09:54:24,746 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackAndDoubleExecution/6b41352f778cf0f0366b57384c1aaf98 2023-02-11 09:54:24,746 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 09:54:24,749 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:24,750 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-11 09:54:24,752 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-11 09:54:24,803 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:24,803 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-11 09:54:24,803 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109264803"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:24,805 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 09:54:24,805 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6b41352f778cf0f0366b57384c1aaf98, NAME => 'testRollbackAndDoubleExecution,,1676109256027.6b41352f778cf0f0366b57384c1aaf98.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 09:54:24,805 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-11 09:54:24,805 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109264805"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:24,806 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-11 09:54:24,809 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:54:24,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 82 msec 2023-02-11 09:54:24,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 09:54:24,833 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-11 09:54:24,847 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=396 (was 400), OpenFileDescriptor=850 (was 867), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 312), ProcessCount=174 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=2747 (was 2892) 2023-02-11 09:54:24,860 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=396, OpenFileDescriptor=850, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=174, AvailableMemoryMB=2746 2023-02-11 09:54:24,861 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:24,863 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:54:24,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:54:24,864 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:24,866 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:54:24,866 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:54:24,868 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:24,868 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7 empty. 2023-02-11 09:54:24,869 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:24,869 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 09:54:24,879 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:24,880 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => b60cbf4c5accba4b5299abb770eaa0f7, NAME => 'testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:54:24,891 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:24,891 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing b60cbf4c5accba4b5299abb770eaa0f7, disabling compactions & flushes 2023-02-11 09:54:24,891 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:24,891 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:24,891 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. after waiting 0 ms 2023-02-11 09:54:24,891 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:24,891 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:24,892 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for b60cbf4c5accba4b5299abb770eaa0f7: Waiting for close lock at 1676109264891Disabling compacts and flushes for region at 1676109264891Disabling writes for close at 1676109264891Writing region close event to WAL at 1676109264891Closed at 1676109264891 2023-02-11 09:54:24,900 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:54:24,901 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676109264901"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109264901"}]},"ts":"1676109264901"} 2023-02-11 09:54:24,903 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:54:24,904 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:54:24,904 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109264904"}]},"ts":"1676109264904"} 2023-02-11 09:54:24,905 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-11 09:54:24,909 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:24,910 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:24,910 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:24,910 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:24,910 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:24,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b60cbf4c5accba4b5299abb770eaa0f7, ASSIGN}] 2023-02-11 09:54:24,912 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b60cbf4c5accba4b5299abb770eaa0f7, ASSIGN 2023-02-11 09:54:24,913 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b60cbf4c5accba4b5299abb770eaa0f7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:54:25,063 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:25,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=b60cbf4c5accba4b5299abb770eaa0f7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676109265065"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109265065"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109265065"}]},"ts":"1676109265065"} 2023-02-11 09:54:25,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure b60cbf4c5accba4b5299abb770eaa0f7, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:25,219 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,219 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 09:54:25,222 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35028, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 09:54:25,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:25,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b60cbf4c5accba4b5299abb770eaa0f7, NAME => 'testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:25,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:25,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:25,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,229 INFO [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,231 DEBUG [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/cf1 2023-02-11 09:54:25,231 DEBUG [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/cf1 2023-02-11 09:54:25,231 INFO [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b60cbf4c5accba4b5299abb770eaa0f7 columnFamilyName cf1 2023-02-11 09:54:25,232 INFO [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] regionserver.HStore(310): Store=b60cbf4c5accba4b5299abb770eaa0f7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:25,232 INFO [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,233 DEBUG [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/cf2 2023-02-11 09:54:25,233 DEBUG [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/cf2 2023-02-11 09:54:25,234 INFO [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b60cbf4c5accba4b5299abb770eaa0f7 columnFamilyName cf2 2023-02-11 09:54:25,234 INFO [StoreOpener-b60cbf4c5accba4b5299abb770eaa0f7-1] regionserver.HStore(310): Store=b60cbf4c5accba4b5299abb770eaa0f7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:25,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:25,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:25,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b60cbf4c5accba4b5299abb770eaa0f7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73302119, jitterRate=0.09228669106960297}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:25,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b60cbf4c5accba4b5299abb770eaa0f7: Running coprocessor pre-open hook at 1676109265228Writing region info on filesystem at 1676109265228Initializing all the Stores at 1676109265229 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109265229Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109265229Cleaning up temporary data from old regions at 1676109265236 (+7 ms)Cleaning up detritus from prior splits at 1676109265237 (+1 ms)Running coprocessor post-open hooks at 1676109265242 (+5 ms)Region opened successfully at 1676109265242 2023-02-11 09:54:25,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7., pid=55, masterSystemTime=1676109265219 2023-02-11 09:54:25,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:25,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:25,247 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=b60cbf4c5accba4b5299abb770eaa0f7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,247 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676109265247"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109265247"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109265247"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109265247"}]},"ts":"1676109265247"} 2023-02-11 09:54:25,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-11 09:54:25,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure b60cbf4c5accba4b5299abb770eaa0f7, server=jenkins-hbase4.apache.org,40095,1676109241701 in 182 msec 2023-02-11 09:54:25,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-11 09:54:25,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b60cbf4c5accba4b5299abb770eaa0f7, ASSIGN in 339 msec 2023-02-11 09:54:25,253 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:54:25,253 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109265253"}]},"ts":"1676109265253"} 2023-02-11 09:54:25,254 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-11 09:54:25,256 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:54:25,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 392 msec 2023-02-11 09:54:25,366 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,367 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,368 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,368 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,368 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b60cbf4c5accba4b5299abb770eaa0f7, UNASSIGN}] 2023-02-11 09:54:25,370 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b60cbf4c5accba4b5299abb770eaa0f7, UNASSIGN 2023-02-11 09:54:25,371 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=b60cbf4c5accba4b5299abb770eaa0f7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,371 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676109265371"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109265371"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109265371"}]},"ts":"1676109265371"} 2023-02-11 09:54:25,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure b60cbf4c5accba4b5299abb770eaa0f7, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:25,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b60cbf4c5accba4b5299abb770eaa0f7, disabling compactions & flushes 2023-02-11 09:54:25,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:25,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:25,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. after waiting 0 ms 2023-02-11 09:54:25,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:25,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 09:54:25,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:25,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7. 2023-02-11 09:54:25,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b60cbf4c5accba4b5299abb770eaa0f7: Waiting for close lock at 1676109265524Running coprocessor pre-close hooks at 1676109265524Disabling compacts and flushes for region at 1676109265524Disabling writes for close at 1676109265525 (+1 ms)Writing region close event to WAL at 1676109265525Running coprocessor post-close hooks at 1676109265529 (+4 ms)Closed at 1676109265529 2023-02-11 09:54:25,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:25,531 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=b60cbf4c5accba4b5299abb770eaa0f7, regionState=CLOSED 2023-02-11 09:54:25,531 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676109265531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109265531"}]},"ts":"1676109265531"} 2023-02-11 09:54:25,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-11 09:54:25,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure b60cbf4c5accba4b5299abb770eaa0f7, server=jenkins-hbase4.apache.org,40095,1676109241701 in 160 msec 2023-02-11 09:54:25,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-11 09:54:25,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b60cbf4c5accba4b5299abb770eaa0f7, UNASSIGN in 165 msec 2023-02-11 09:54:25,559 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 09:54:25,560 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 09:54:25,562 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676109265562"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676109265562"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676109265562"}]},"ts":"1676109265562"} 2023-02-11 09:54:25,562 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109265562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109265562"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109265562"}]},"ts":"1676109265562"} 2023-02-11 09:54:25,562 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109265562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109265562"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109265562"}]},"ts":"1676109265562"} 2023-02-11 09:54:25,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=199774c1460010773c61876a45accec6, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7d4c56edfb8f163faef5374f2d1354c4, ASSIGN}] 2023-02-11 09:54:25,565 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7d4c56edfb8f163faef5374f2d1354c4, ASSIGN 2023-02-11 09:54:25,566 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7d4c56edfb8f163faef5374f2d1354c4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:54:25,566 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=199774c1460010773c61876a45accec6, ASSIGN 2023-02-11 09:54:25,567 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=199774c1460010773c61876a45accec6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:54:25,716 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 09:54:25,717 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=199774c1460010773c61876a45accec6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,718 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109265717"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109265717"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109265717"}]},"ts":"1676109265717"} 2023-02-11 09:54:25,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 199774c1460010773c61876a45accec6, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:25,721 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=7d4c56edfb8f163faef5374f2d1354c4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,722 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109265721"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109265721"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109265721"}]},"ts":"1676109265721"} 2023-02-11 09:54:25,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 7d4c56edfb8f163faef5374f2d1354c4, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:25,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. 2023-02-11 09:54:25,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7d4c56edfb8f163faef5374f2d1354c4, NAME => 'testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 09:54:25,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:25,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:25,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,875 INFO [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,877 DEBUG [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/cf1 2023-02-11 09:54:25,877 DEBUG [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/cf1 2023-02-11 09:54:25,877 INFO [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7d4c56edfb8f163faef5374f2d1354c4 columnFamilyName cf1 2023-02-11 09:54:25,878 INFO [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] regionserver.HStore(310): Store=7d4c56edfb8f163faef5374f2d1354c4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:25,878 INFO [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,879 DEBUG [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/cf2 2023-02-11 09:54:25,879 DEBUG [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/cf2 2023-02-11 09:54:25,879 INFO [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7d4c56edfb8f163faef5374f2d1354c4 columnFamilyName cf2 2023-02-11 09:54:25,880 INFO [StoreOpener-7d4c56edfb8f163faef5374f2d1354c4-1] regionserver.HStore(310): Store=7d4c56edfb8f163faef5374f2d1354c4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:25,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:25,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:25,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7d4c56edfb8f163faef5374f2d1354c4; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60231689, jitterRate=-0.10247789323329926}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:25,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7d4c56edfb8f163faef5374f2d1354c4: Running coprocessor pre-open hook at 1676109265874Writing region info on filesystem at 1676109265874Initializing all the Stores at 1676109265875 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109265875Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109265875Cleaning up temporary data from old regions at 1676109265884 (+9 ms)Cleaning up detritus from prior splits at 1676109265884Running coprocessor post-open hooks at 1676109265887 (+3 ms)Region opened successfully at 1676109265887 2023-02-11 09:54:25,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4., pid=62, masterSystemTime=1676109265871 2023-02-11 09:54:25,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. 2023-02-11 09:54:25,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. 2023-02-11 09:54:25,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. 2023-02-11 09:54:25,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 199774c1460010773c61876a45accec6, NAME => 'testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 09:54:25,890 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=7d4c56edfb8f163faef5374f2d1354c4, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:25,890 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109265890"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109265890"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109265890"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109265890"}]},"ts":"1676109265890"} 2023-02-11 09:54:25,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 199774c1460010773c61876a45accec6 2023-02-11 09:54:25,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:25,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 199774c1460010773c61876a45accec6 2023-02-11 09:54:25,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 199774c1460010773c61876a45accec6 2023-02-11 09:54:25,891 INFO [StoreOpener-199774c1460010773c61876a45accec6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 199774c1460010773c61876a45accec6 2023-02-11 09:54:25,893 DEBUG [StoreOpener-199774c1460010773c61876a45accec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/cf1 2023-02-11 09:54:25,893 DEBUG [StoreOpener-199774c1460010773c61876a45accec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/cf1 2023-02-11 09:54:25,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-11 09:54:25,893 INFO [StoreOpener-199774c1460010773c61876a45accec6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 199774c1460010773c61876a45accec6 columnFamilyName cf1 2023-02-11 09:54:25,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 7d4c56edfb8f163faef5374f2d1354c4, server=jenkins-hbase4.apache.org,40095,1676109241701 in 169 msec 2023-02-11 09:54:25,894 INFO [StoreOpener-199774c1460010773c61876a45accec6-1] regionserver.HStore(310): Store=199774c1460010773c61876a45accec6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:25,894 INFO [StoreOpener-199774c1460010773c61876a45accec6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 199774c1460010773c61876a45accec6 2023-02-11 09:54:25,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7d4c56edfb8f163faef5374f2d1354c4, ASSIGN in 330 msec 2023-02-11 09:54:25,895 DEBUG [StoreOpener-199774c1460010773c61876a45accec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/cf2 2023-02-11 09:54:25,896 DEBUG [StoreOpener-199774c1460010773c61876a45accec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/cf2 2023-02-11 09:54:25,896 INFO [StoreOpener-199774c1460010773c61876a45accec6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 199774c1460010773c61876a45accec6 columnFamilyName cf2 2023-02-11 09:54:25,896 INFO [StoreOpener-199774c1460010773c61876a45accec6-1] regionserver.HStore(310): Store=199774c1460010773c61876a45accec6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:25,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6 2023-02-11 09:54:25,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6 2023-02-11 09:54:25,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:25,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 199774c1460010773c61876a45accec6 2023-02-11 09:54:25,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 199774c1460010773c61876a45accec6; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65491161, jitterRate=-0.024105653166770935}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:25,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 199774c1460010773c61876a45accec6: Running coprocessor pre-open hook at 1676109265890Writing region info on filesystem at 1676109265890Initializing all the Stores at 1676109265891 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109265891Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109265891Cleaning up temporary data from old regions at 1676109265899 (+8 ms)Cleaning up detritus from prior splits at 1676109265899Running coprocessor post-open hooks at 1676109265902 (+3 ms)Region opened successfully at 1676109265902 2023-02-11 09:54:25,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6., pid=61, masterSystemTime=1676109265871 2023-02-11 09:54:25,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. 2023-02-11 09:54:25,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. 2023-02-11 09:54:25,905 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=199774c1460010773c61876a45accec6, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:25,905 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109265905"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109265905"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109265905"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109265905"}]},"ts":"1676109265905"} 2023-02-11 09:54:25,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-11 09:54:25,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 199774c1460010773c61876a45accec6, server=jenkins-hbase4.apache.org,40095,1676109241701 in 188 msec 2023-02-11 09:54:25,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-11 09:54:25,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=199774c1460010773c61876a45accec6, ASSIGN in 344 msec 2023-02-11 09:54:25,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 in 544 msec 2023-02-11 09:54:26,120 DEBUG [hconnection-0x7e2f5f25-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 09:54:26,124 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35030, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 09:54:26,125 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 09:54:26,128 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:26,129 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 09:54:26,133 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109266133"}]},"ts":"1676109266133"} 2023-02-11 09:54:26,134 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-11 09:54:26,137 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-11 09:54:26,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=199774c1460010773c61876a45accec6, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7d4c56edfb8f163faef5374f2d1354c4, UNASSIGN}] 2023-02-11 09:54:26,138 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7d4c56edfb8f163faef5374f2d1354c4, UNASSIGN 2023-02-11 09:54:26,139 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=7d4c56edfb8f163faef5374f2d1354c4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:26,139 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109266139"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109266139"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109266139"}]},"ts":"1676109266139"} 2023-02-11 09:54:26,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 7d4c56edfb8f163faef5374f2d1354c4, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:26,142 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=199774c1460010773c61876a45accec6, UNASSIGN 2023-02-11 09:54:26,143 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=199774c1460010773c61876a45accec6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:26,143 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109266143"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109266143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109266143"}]},"ts":"1676109266143"} 2023-02-11 09:54:26,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 199774c1460010773c61876a45accec6, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:26,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 09:54:26,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 199774c1460010773c61876a45accec6 2023-02-11 09:54:26,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 199774c1460010773c61876a45accec6, disabling compactions & flushes 2023-02-11 09:54:26,293 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. 2023-02-11 09:54:26,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. 2023-02-11 09:54:26,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. after waiting 0 ms 2023-02-11 09:54:26,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. 2023-02-11 09:54:26,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 09:54:26,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:26,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6. 2023-02-11 09:54:26,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 199774c1460010773c61876a45accec6: Waiting for close lock at 1676109266292Running coprocessor pre-close hooks at 1676109266292Disabling compacts and flushes for region at 1676109266292Disabling writes for close at 1676109266293 (+1 ms)Writing region close event to WAL at 1676109266293Running coprocessor post-close hooks at 1676109266297 (+4 ms)Closed at 1676109266297 2023-02-11 09:54:26,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 199774c1460010773c61876a45accec6 2023-02-11 09:54:26,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:26,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7d4c56edfb8f163faef5374f2d1354c4, disabling compactions & flushes 2023-02-11 09:54:26,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. 2023-02-11 09:54:26,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. 2023-02-11 09:54:26,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. after waiting 0 ms 2023-02-11 09:54:26,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. 2023-02-11 09:54:26,300 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=199774c1460010773c61876a45accec6, regionState=CLOSED 2023-02-11 09:54:26,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109266300"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109266300"}]},"ts":"1676109266300"} 2023-02-11 09:54:26,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-11 09:54:26,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 199774c1460010773c61876a45accec6, server=jenkins-hbase4.apache.org,40095,1676109241701 in 157 msec 2023-02-11 09:54:26,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 09:54:26,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:26,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4. 2023-02-11 09:54:26,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7d4c56edfb8f163faef5374f2d1354c4: Waiting for close lock at 1676109266299Running coprocessor pre-close hooks at 1676109266299Disabling compacts and flushes for region at 1676109266299Disabling writes for close at 1676109266299Writing region close event to WAL at 1676109266300 (+1 ms)Running coprocessor post-close hooks at 1676109266304 (+4 ms)Closed at 1676109266304 2023-02-11 09:54:26,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=199774c1460010773c61876a45accec6, UNASSIGN in 166 msec 2023-02-11 09:54:26,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:26,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=7d4c56edfb8f163faef5374f2d1354c4, regionState=CLOSED 2023-02-11 09:54:26,306 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109266305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109266305"}]},"ts":"1676109266305"} 2023-02-11 09:54:26,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-11 09:54:26,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 7d4c56edfb8f163faef5374f2d1354c4, server=jenkins-hbase4.apache.org,40095,1676109241701 in 167 msec 2023-02-11 09:54:26,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-11 09:54:26,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7d4c56edfb8f163faef5374f2d1354c4, UNASSIGN in 171 msec 2023-02-11 09:54:26,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109266310"}]},"ts":"1676109266310"} 2023-02-11 09:54:26,311 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-11 09:54:26,312 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-11 09:54:26,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 183 msec 2023-02-11 09:54:26,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 09:54:26,435 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-11 09:54:26,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,438 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 09:54:26,439 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,444 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:26,445 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6 2023-02-11 09:54:26,445 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:26,448 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/recovered.edits] 2023-02-11 09:54:26,448 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/recovered.edits] 2023-02-11 09:54:26,449 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/recovered.edits] 2023-02-11 09:54:26,460 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/recovered.edits/4.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7/recovered.edits/4.seqid 2023-02-11 09:54:26,461 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/recovered.edits/7.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4/recovered.edits/7.seqid 2023-02-11 09:54:26,461 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/recovered.edits/7.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6/recovered.edits/7.seqid 2023-02-11 09:54:26,461 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/b60cbf4c5accba4b5299abb770eaa0f7 2023-02-11 09:54:26,462 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/199774c1460010773c61876a45accec6 2023-02-11 09:54:26,462 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionNoStoreFile/7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:54:26,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 09:54:26,465 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,466 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-11 09:54:26,468 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-11 09:54:26,469 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,469 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-11 09:54:26,469 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109266469"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:26,470 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109266469"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:26,470 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109266469"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:26,471 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 09:54:26,471 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b60cbf4c5accba4b5299abb770eaa0f7, NAME => 'testSplitTableRegionNoStoreFile,,1676109264864.b60cbf4c5accba4b5299abb770eaa0f7.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 199774c1460010773c61876a45accec6, NAME => 'testSplitTableRegionNoStoreFile,,1676109265366.199774c1460010773c61876a45accec6.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 7d4c56edfb8f163faef5374f2d1354c4, NAME => 'testSplitTableRegionNoStoreFile,41,1676109265366.7d4c56edfb8f163faef5374f2d1354c4.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 09:54:26,471 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-11 09:54:26,471 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109266471"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:26,473 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-11 09:54:26,475 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:54:26,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 40 msec 2023-02-11 09:54:26,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 09:54:26,540 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-11 09:54:26,555 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406 (was 396) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-494796024_15 at /127.0.0.1:57770 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-494796024_15 at /127.0.0.1:54576 [Waiting for operation #40] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=856 (was 850) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 271), ProcessCount=175 (was 174) - ProcessCount LEAK? -, AvailableMemoryMB=2742 (was 2746) 2023-02-11 09:54:26,569 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406, OpenFileDescriptor=856, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=175, AvailableMemoryMB=2742 2023-02-11 09:54:26,569 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:26,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:54:26,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:54:26,574 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:26,575 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:54:26,576 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:54:26,578 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,578 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8 empty. 2023-02-11 09:54:26,579 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,579 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 09:54:26,597 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:26,598 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => a6cffd29aa384a14f910e924d07d1ce8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:54:26,607 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:26,607 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing a6cffd29aa384a14f910e924d07d1ce8, disabling compactions & flushes 2023-02-11 09:54:26,607 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:26,607 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:26,607 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. after waiting 0 ms 2023-02-11 09:54:26,607 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:26,607 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:26,607 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for a6cffd29aa384a14f910e924d07d1ce8: Waiting for close lock at 1676109266607Disabling compacts and flushes for region at 1676109266607Disabling writes for close at 1676109266607Writing region close event to WAL at 1676109266607Closed at 1676109266607 2023-02-11 09:54:26,609 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:54:26,610 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676109266610"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109266610"}]},"ts":"1676109266610"} 2023-02-11 09:54:26,611 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:54:26,612 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:54:26,612 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109266612"}]},"ts":"1676109266612"} 2023-02-11 09:54:26,613 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-11 09:54:26,617 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:26,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:26,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:26,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:26,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:26,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a6cffd29aa384a14f910e924d07d1ce8, ASSIGN}] 2023-02-11 09:54:26,619 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a6cffd29aa384a14f910e924d07d1ce8, ASSIGN 2023-02-11 09:54:26,620 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a6cffd29aa384a14f910e924d07d1ce8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:54:26,771 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:26,772 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=a6cffd29aa384a14f910e924d07d1ce8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:26,772 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676109266772"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109266772"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109266772"}]},"ts":"1676109266772"} 2023-02-11 09:54:26,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure a6cffd29aa384a14f910e924d07d1ce8, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:26,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:26,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6cffd29aa384a14f910e924d07d1ce8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:26,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:26,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:26,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,930 INFO [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,931 DEBUG [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1 2023-02-11 09:54:26,931 DEBUG [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1 2023-02-11 09:54:26,931 INFO [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6cffd29aa384a14f910e924d07d1ce8 columnFamilyName cf1 2023-02-11 09:54:26,932 INFO [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] regionserver.HStore(310): Store=a6cffd29aa384a14f910e924d07d1ce8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:26,932 INFO [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,933 DEBUG [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2 2023-02-11 09:54:26,933 DEBUG [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2 2023-02-11 09:54:26,933 INFO [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6cffd29aa384a14f910e924d07d1ce8 columnFamilyName cf2 2023-02-11 09:54:26,934 INFO [StoreOpener-a6cffd29aa384a14f910e924d07d1ce8-1] regionserver.HStore(310): Store=a6cffd29aa384a14f910e924d07d1ce8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:26,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:26,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:26,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:26,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a6cffd29aa384a14f910e924d07d1ce8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62996592, jitterRate=-0.06127762794494629}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:26,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a6cffd29aa384a14f910e924d07d1ce8: Running coprocessor pre-open hook at 1676109266929Writing region info on filesystem at 1676109266929Initializing all the Stores at 1676109266929Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109266929Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109266929Cleaning up temporary data from old regions at 1676109266935 (+6 ms)Cleaning up detritus from prior splits at 1676109266936 (+1 ms)Running coprocessor post-open hooks at 1676109266941 (+5 ms)Region opened successfully at 1676109266941 2023-02-11 09:54:26,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8., pid=71, masterSystemTime=1676109266925 2023-02-11 09:54:26,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:26,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:26,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=a6cffd29aa384a14f910e924d07d1ce8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:26,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676109266944"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109266944"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109266944"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109266944"}]},"ts":"1676109266944"} 2023-02-11 09:54:26,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-11 09:54:26,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure a6cffd29aa384a14f910e924d07d1ce8, server=jenkins-hbase4.apache.org,40095,1676109241701 in 172 msec 2023-02-11 09:54:26,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-11 09:54:26,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a6cffd29aa384a14f910e924d07d1ce8, ASSIGN in 329 msec 2023-02-11 09:54:26,949 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:54:26,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109266949"}]},"ts":"1676109266949"} 2023-02-11 09:54:26,951 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-11 09:54:26,953 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:54:26,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 380 msec 2023-02-11 09:54:27,043 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:27,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,083 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:27,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:27,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,084 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,085 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,085 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,085 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,085 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,085 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,087 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:27,087 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,087 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:27,087 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,087 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,087 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,087 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,087 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:27,088 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:54:27,089 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,089 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,089 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,089 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,089 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,089 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,104 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/bd79b9e52b6e49949176a4acd45db180 2023-02-11 09:54:27,124 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/5f208f1ba251438793cb590f70677ef2 2023-02-11 09:54:27,129 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/bd79b9e52b6e49949176a4acd45db180 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180 2023-02-11 09:54:27,133 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:27,134 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/5f208f1ba251438793cb590f70677ef2 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2 2023-02-11 09:54:27,137 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:54:27,138 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 50ms, sequenceid=6, compaction requested=false 2023-02-11 09:54:27,139 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:27,139 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,139 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:27,140 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:27,140 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,140 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,140 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,141 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,142 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,142 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41abbd5a[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:27,144 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:27,144 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,146 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,146 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,146 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,147 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,147 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,147 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:27,149 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:27,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:27,153 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:27,153 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:27,153 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:27,153 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,153 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:27,155 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,155 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,155 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,156 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,156 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,157 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,158 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,158 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,159 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:27,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,166 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,165 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,165 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,166 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:27,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,166 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:27,166 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,166 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,167 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,167 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,167 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,167 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,266 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:27,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:27,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:27,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:27,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,282 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,282 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,282 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:27,282 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:27,282 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,282 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,283 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,283 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,284 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,283 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,284 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,284 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:27,286 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:27,286 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,286 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:27,287 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:27,287 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,288 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,288 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,288 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,288 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,288 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,308 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/fe5da30219684eb88f983578dde96f9f 2023-02-11 09:54:27,329 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/99d47d4616e343dd993b370bc8306f7f 2023-02-11 09:54:27,334 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/fe5da30219684eb88f983578dde96f9f as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f 2023-02-11 09:54:27,338 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:27,338 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/99d47d4616e343dd993b370bc8306f7f as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f 2023-02-11 09:54:27,343 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:54:27,344 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 56ms, sequenceid=19, compaction requested=false 2023-02-11 09:54:27,344 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:27,344 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,347 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,347 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,347 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,347 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d7c183f[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:27,349 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:27,349 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,350 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,350 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:27,351 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:27,351 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:27,352 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,352 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,353 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,353 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,353 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,353 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,353 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,353 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,354 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,354 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:27,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:27,357 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:27,357 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:27,357 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:27,357 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,357 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:27,360 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,360 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,360 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,360 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,361 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,362 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,363 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,364 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,364 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,367 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:27,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:27,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,367 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,368 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:27,468 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:27,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:27,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:27,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:27,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,485 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,485 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,485 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:27,485 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:27,485 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,485 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,488 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,488 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,488 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,488 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,488 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,489 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:27,491 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:27,491 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,491 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:27,491 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:27,492 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,493 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,493 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,493 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,493 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,493 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,503 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/b47de98067034be1b5dd09cd10e3879e 2023-02-11 09:54:27,522 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/0f1d224c52244d868e8fc16eae295858 2023-02-11 09:54:27,527 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/b47de98067034be1b5dd09cd10e3879e as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e 2023-02-11 09:54:27,532 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:27,533 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/0f1d224c52244d868e8fc16eae295858 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858 2023-02-11 09:54:27,537 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:54:27,539 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 48ms, sequenceid=32, compaction requested=true 2023-02-11 09:54:27,539 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:27,539 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,539 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:27,539 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:27,539 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,539 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,539 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,544 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,544 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,545 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,545 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:27,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c5a9592[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:27,547 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:27,547 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,548 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,548 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,548 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,548 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,549 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:27,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,549 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:27,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,549 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,549 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,549 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,549 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,549 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:27,549 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:27,550 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,550 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,550 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,551 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,551 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,551 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,551 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,551 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,551 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,551 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:27,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:27,555 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:27,555 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:27,555 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:27,555 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,555 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:27,561 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,561 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,561 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,562 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,562 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,562 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,563 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,566 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,566 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,566 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,570 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:27,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,570 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:27,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,571 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,671 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:27,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:27,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:27,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:27,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,687 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,687 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,687 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:27,687 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:27,688 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,688 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,689 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,689 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,689 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,689 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,690 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,690 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:27,692 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,692 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:27,692 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:27,694 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,695 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,695 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,695 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,695 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,696 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,708 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/625f7386e776487aa1388d1d7fb926fd 2023-02-11 09:54:27,720 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/41e82de59ed5419aa97d5aea99e71a46 2023-02-11 09:54:27,724 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/625f7386e776487aa1388d1d7fb926fd as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/625f7386e776487aa1388d1d7fb926fd 2023-02-11 09:54:27,728 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/625f7386e776487aa1388d1d7fb926fd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:27,729 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/41e82de59ed5419aa97d5aea99e71a46 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/41e82de59ed5419aa97d5aea99e71a46 2023-02-11 09:54:27,733 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/41e82de59ed5419aa97d5aea99e71a46, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:54:27,734 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 42ms, sequenceid=45, compaction requested=true 2023-02-11 09:54:27,734 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:27,734 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,734 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:27,734 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:27,734 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,734 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,734 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,736 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,736 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,736 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,736 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:27,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23facda6[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:27,738 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:27,738 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,739 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:27,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:27,742 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,742 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,742 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,742 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,742 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,743 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,743 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,743 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,743 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,743 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:27,746 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:27,746 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:27,746 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:27,746 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,746 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:27,750 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,750 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,750 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,751 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,751 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,751 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,752 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,753 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,753 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,754 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,757 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,757 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,758 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,858 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:27,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:27,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:27,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:27,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,871 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,872 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,872 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:27,872 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:27,872 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,872 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,873 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,873 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,873 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,873 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,874 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,874 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,875 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,875 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,875 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,875 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:27,875 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,876 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:27,876 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,878 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,878 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,884 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/2f268b53a3da42da966de356108d75a0 2023-02-11 09:54:27,897 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/411aa2a87b234c4493e1c35e77a09b9a 2023-02-11 09:54:27,902 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/2f268b53a3da42da966de356108d75a0 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2f268b53a3da42da966de356108d75a0 2023-02-11 09:54:27,906 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2f268b53a3da42da966de356108d75a0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:27,906 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/411aa2a87b234c4493e1c35e77a09b9a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/411aa2a87b234c4493e1c35e77a09b9a 2023-02-11 09:54:27,910 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/411aa2a87b234c4493e1c35e77a09b9a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:54:27,911 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 35ms, sequenceid=58, compaction requested=true 2023-02-11 09:54:27,911 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:27,911 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:27,911 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:27,911 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:27,911 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,911 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:27,911 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:27,914 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,914 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,914 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,914 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:27,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f27f5ee[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:27,916 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:27,916 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,917 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,917 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,917 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,918 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,918 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:27,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:27,918 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:27,919 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,919 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,919 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:27,924 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:27,928 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:27,928 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:27,928 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:27,928 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,928 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:27,931 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,931 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:27,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,932 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:27,932 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:27,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:27,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,934 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,934 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,935 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:27,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,937 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,937 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,938 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:27,937 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,937 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,937 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:27,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:27,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter because node does not exist (not an error) 2023-02-11 09:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,938 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:27,939 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:27,939 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:27,939 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:27,939 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:27,939 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,039 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:28,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:28,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:28,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:28,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,052 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:28,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:28,052 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,053 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,056 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,056 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,057 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,058 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,059 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:28,059 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:28,059 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,059 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:28,059 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:54:28,061 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,061 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,061 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,061 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,062 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,062 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,068 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/b908652a136646d986886d9a4fd8c3c6 2023-02-11 09:54:28,081 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/c7ab338240bf48728e98e93a5227e59a 2023-02-11 09:54:28,085 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/b908652a136646d986886d9a4fd8c3c6 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b908652a136646d986886d9a4fd8c3c6 2023-02-11 09:54:28,088 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b908652a136646d986886d9a4fd8c3c6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:28,089 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/c7ab338240bf48728e98e93a5227e59a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/c7ab338240bf48728e98e93a5227e59a 2023-02-11 09:54:28,093 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/c7ab338240bf48728e98e93a5227e59a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:54:28,094 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 35ms, sequenceid=71, compaction requested=true 2023-02-11 09:54:28,094 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:28,094 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,096 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,096 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,096 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,096 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:28,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2287afd5[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:28,098 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:28,098 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,100 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,100 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,100 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:28,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,104 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,104 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,105 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,105 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,105 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,105 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,105 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:28,108 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:28,108 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:28,108 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:28,108 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,108 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:28,110 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,110 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,111 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,111 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,111 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,113 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,113 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,113 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,117 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:28,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:28,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,218 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:28,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:28,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:28,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:28,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,232 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,232 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,232 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:28,232 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:28,233 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,233 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,234 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,234 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,234 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,234 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,235 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,236 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,237 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:28,237 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:28,237 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,237 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:28,237 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-11 09:54:28,238 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,238 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,238 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,238 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,238 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,238 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,247 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/4327659ae8514a3abfbbed93376aa59b 2023-02-11 09:54:28,251 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4327659ae8514a3abfbbed93376aa59b 2023-02-11 09:54:28,258 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/d3bd1dc076a642c8adbbb4b9971f465c 2023-02-11 09:54:28,261 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bd1dc076a642c8adbbb4b9971f465c 2023-02-11 09:54:28,262 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/4327659ae8514a3abfbbed93376aa59b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b 2023-02-11 09:54:28,265 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4327659ae8514a3abfbbed93376aa59b 2023-02-11 09:54:28,265 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 09:54:28,266 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/d3bd1dc076a642c8adbbb4b9971f465c as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c 2023-02-11 09:54:28,269 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bd1dc076a642c8adbbb4b9971f465c 2023-02-11 09:54:28,269 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 09:54:28,270 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 33ms, sequenceid=83, compaction requested=true 2023-02-11 09:54:28,270 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:28,270 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,270 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:28,270 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:28,270 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,270 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,270 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,271 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,271 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,272 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,272 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:28,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a095812[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:28,273 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:28,274 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,280 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:28,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,280 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:28,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:28,283 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,283 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,288 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,288 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,288 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,288 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,288 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,288 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:28,290 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:28,290 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:28,290 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:28,290 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,290 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:28,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,293 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,294 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,294 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,294 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,296 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,296 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,296 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:28,299 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,300 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:28,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,300 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:28,300 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,300 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,300 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,300 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,400 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:28,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:28,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:28,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:28,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,411 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:28,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:28,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,413 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,413 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,413 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:28,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,414 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,414 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:28,415 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,415 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:28,415 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 09:54:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,424 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:28,427 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:28,435 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:28,438 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:28,438 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/88a7ccb96c1240edbde24c468cbdcdd3 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:28,442 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:28,442 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/88a7ccb96c1240edbde24c468cbdcdd3, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 09:54:28,443 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:28,446 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:28,446 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 09:54:28,447 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 32ms, sequenceid=91, compaction requested=true 2023-02-11 09:54:28,447 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:28,447 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,449 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,449 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,449 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,449 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:28,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2feb3908[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:28,451 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:28,451 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,452 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,452 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,454 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:28,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:28,457 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:28,457 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:28,457 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:28,457 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,457 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:28,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,459 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,460 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,460 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,461 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,462 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,462 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,462 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,467 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,467 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,468 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:28,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,469 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:28,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,469 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,569 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:28,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:28,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:54:28,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:54:28,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,579 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,580 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,580 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 09:54:28,580 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 09:54:28,580 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,580 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,583 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,583 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,583 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,583 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,583 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,583 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:54:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,584 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,585 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,585 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 09:54:28,585 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 09:54:28,585 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. started... 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,585 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,585 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 09:54:28,586 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,587 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,587 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,587 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,588 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,588 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,594 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:28,598 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:28,606 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/e034bbe3a64b456797baac442725082f 2023-02-11 09:54:28,609 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e034bbe3a64b456797baac442725082f 2023-02-11 09:54:28,610 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/2011e7580cb04c1da3a10f90f0d5219b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:28,614 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:28,614 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2011e7580cb04c1da3a10f90f0d5219b, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 09:54:28,615 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/e034bbe3a64b456797baac442725082f as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/e034bbe3a64b456797baac442725082f 2023-02-11 09:54:28,619 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e034bbe3a64b456797baac442725082f 2023-02-11 09:54:28,619 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/e034bbe3a64b456797baac442725082f, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 09:54:28,620 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 35ms, sequenceid=99, compaction requested=true 2023-02-11 09:54:28,620 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6cffd29aa384a14f910e924d07d1ce8: 2023-02-11 09:54:28,620 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,620 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:54:28,620 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:54:28,620 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,620 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 09:54:28,620 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 09:54:28,622 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,622 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,622 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,622 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:54:28,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 09:54:28,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f63319e[Count = 0] remaining members to acquire global barrier 2023-02-11 09:54:28,624 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 09:54:28,624 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,626 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,626 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,626 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:54:28,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,626 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:54:28,626 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 09:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 09:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:54:28,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,628 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,629 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,629 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,629 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,629 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,629 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 09:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,629 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,630 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 09:54:28,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:54:28,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:54:28,632 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 09:54:28,632 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:54:28,632 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:54:28,632 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,632 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:54:28,635 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,635 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,635 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,635 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,635 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,636 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,636 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:54:28,636 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,637 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,637 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:54:28,639 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,639 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,639 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,641 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,641 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,641 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,641 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,642 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 09:54:28,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:54:28,641 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:54:28,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,642 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:54:28,643 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:28,643 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:54:28,643 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,643 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:28,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:54:28,742 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:54:28,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:54:28,745 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,746 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:28,747 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:28,747 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:28,747 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:28,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a6cffd29aa384a14f910e924d07d1ce8, UNASSIGN}] 2023-02-11 09:54:28,750 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a6cffd29aa384a14f910e924d07d1ce8, UNASSIGN 2023-02-11 09:54:28,750 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=a6cffd29aa384a14f910e924d07d1ce8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:28,750 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676109268750"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109268750"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109268750"}]},"ts":"1676109268750"} 2023-02-11 09:54:28,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure a6cffd29aa384a14f910e924d07d1ce8, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:28,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a6cffd29aa384a14f910e924d07d1ce8, disabling compactions & flushes 2023-02-11 09:54:28,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. after waiting 0 ms 2023-02-11 09:54:28,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a6cffd29aa384a14f910e924d07d1ce8 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-11 09:54:28,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:28,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:28,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:28,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:28,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf1/d6f1bde2b0d6406399204149d5e084d9 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:28,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:28,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/d6f1bde2b0d6406399204149d5e084d9, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 09:54:28,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.tmp/cf2/3b50b563a593438bad2e04729c60cee6 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:28,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:28,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/3b50b563a593438bad2e04729c60cee6, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 09:54:28,939 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 36ms, sequenceid=103, compaction requested=true 2023-02-11 09:54:28,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-11 09:54:28,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:28,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. 2023-02-11 09:54:28,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a6cffd29aa384a14f910e924d07d1ce8: Waiting for close lock at 1676109268903Running coprocessor pre-close hooks at 1676109268903Disabling compacts and flushes for region at 1676109268903Disabling writes for close at 1676109268903Obtaining lock to block concurrent updates at 1676109268903Preparing flush snapshotting stores in a6cffd29aa384a14f910e924d07d1ce8 at 1676109268903Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676109268903Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8. at 1676109268904 (+1 ms)Flushing a6cffd29aa384a14f910e924d07d1ce8/cf1: creating writer at 1676109268905 (+1 ms)Flushing a6cffd29aa384a14f910e924d07d1ce8/cf1: appending metadata at 1676109268907 (+2 ms)Flushing a6cffd29aa384a14f910e924d07d1ce8/cf1: closing flushed file at 1676109268907Flushing a6cffd29aa384a14f910e924d07d1ce8/cf2: creating writer at 1676109268918 (+11 ms)Flushing a6cffd29aa384a14f910e924d07d1ce8/cf2: appending metadata at 1676109268919 (+1 ms)Flushing a6cffd29aa384a14f910e924d07d1ce8/cf2: closing flushed file at 1676109268919Flushing a6cffd29aa384a14f910e924d07d1ce8/cf1: reopening flushed file at 1676109268930 (+11 ms)Flushing a6cffd29aa384a14f910e924d07d1ce8/cf2: reopening flushed file at 1676109268935 (+5 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for a6cffd29aa384a14f910e924d07d1ce8 in 36ms, sequenceid=103, compaction requested=true at 1676109268939 (+4 ms)Writing region close event to WAL at 1676109268941 (+2 ms)Running coprocessor post-close hooks at 1676109268944 (+3 ms)Closed at 1676109268944 2023-02-11 09:54:28,945 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,946 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=a6cffd29aa384a14f910e924d07d1ce8, regionState=CLOSED 2023-02-11 09:54:28,946 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676109268946"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109268946"}]},"ts":"1676109268946"} 2023-02-11 09:54:28,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-11 09:54:28,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure a6cffd29aa384a14f910e924d07d1ce8, server=jenkins-hbase4.apache.org,40095,1676109241701 in 196 msec 2023-02-11 09:54:28,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-11 09:54:28,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a6cffd29aa384a14f910e924d07d1ce8, UNASSIGN in 199 msec 2023-02-11 09:54:28,962 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=a6cffd29aa384a14f910e924d07d1ce8, threads=16 2023-02-11 09:54:28,962 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,962 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,963 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/411aa2a87b234c4493e1c35e77a09b9a for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,963 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/3b50b563a593438bad2e04729c60cee6 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,967 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/41e82de59ed5419aa97d5aea99e71a46 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,967 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,967 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,968 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/c7ab338240bf48728e98e93a5227e59a for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,968 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,972 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/e034bbe3a64b456797baac442725082f for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,972 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2011e7580cb04c1da3a10f90f0d5219b for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,972 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2f268b53a3da42da966de356108d75a0 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,972 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,973 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/625f7386e776487aa1388d1d7fb926fd for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,974 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/88a7ccb96c1240edbde24c468cbdcdd3 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,974 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:28,980 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/625f7386e776487aa1388d1d7fb926fd, top=false 2023-02-11 09:54:28,981 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/411aa2a87b234c4493e1c35e77a09b9a, top=false 2023-02-11 09:54:28,981 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:28,982 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e034bbe3a64b456797baac442725082f 2023-02-11 09:54:28,984 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:28,986 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/41e82de59ed5419aa97d5aea99e71a46, top=false 2023-02-11 09:54:28,987 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2f268b53a3da42da966de356108d75a0, top=false 2023-02-11 09:54:28,987 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e034bbe3a64b456797baac442725082f 2023-02-11 09:54:28,987 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/e034bbe3a64b456797baac442725082f, top=true 2023-02-11 09:54:28,989 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:28,991 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:28,991 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1, top=true 2023-02-11 09:54:28,991 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/c7ab338240bf48728e98e93a5227e59a, top=false 2023-02-11 09:54:28,991 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:28,992 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4327659ae8514a3abfbbed93376aa59b 2023-02-11 09:54:28,992 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bd1dc076a642c8adbbb4b9971f465c 2023-02-11 09:54:28,994 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:28,994 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/88a7ccb96c1240edbde24c468cbdcdd3, top=true 2023-02-11 09:54:28,998 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:28,998 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/3b50b563a593438bad2e04729c60cee6, top=true 2023-02-11 09:54:28,999 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:28,999 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2011e7580cb04c1da3a10f90f0d5219b, top=true 2023-02-11 09:54:29,016 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-41e82de59ed5419aa97d5aea99e71a46 for child: 79bd21cbba42e27ce102925ee9dc168a, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,020 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/41e82de59ed5419aa97d5aea99e71a46 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,021 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b908652a136646d986886d9a4fd8c3c6 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,023 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b908652a136646d986886d9a4fd8c3c6, top=false 2023-02-11 09:54:29,024 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-126ebc0a0ed640f7886b8e7ea34cc0b1 for child: e42933689e13bfee2ea375c3661dba49, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,024 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,024 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2f268b53a3da42da966de356108d75a0 for child: 79bd21cbba42e27ce102925ee9dc168a, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,024 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-e034bbe3a64b456797baac442725082f for child: e42933689e13bfee2ea375c3661dba49, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,024 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-625f7386e776487aa1388d1d7fb926fd for child: 79bd21cbba42e27ce102925ee9dc168a, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,024 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/e034bbe3a64b456797baac442725082f for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,024 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,024 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/d6f1bde2b0d6406399204149d5e084d9 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,028 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-88a7ccb96c1240edbde24c468cbdcdd3 for child: e42933689e13bfee2ea375c3661dba49, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,028 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/88a7ccb96c1240edbde24c468cbdcdd3 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,028 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,028 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-c7ab338240bf48728e98e93a5227e59a for child: 79bd21cbba42e27ce102925ee9dc168a, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,029 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2f268b53a3da42da966de356108d75a0 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,029 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:29,032 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/c7ab338240bf48728e98e93a5227e59a for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,033 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/625f7386e776487aa1388d1d7fb926fd for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,033 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:29,034 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/d6f1bde2b0d6406399204149d5e084d9, top=true 2023-02-11 09:54:29,059 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-411aa2a87b234c4493e1c35e77a09b9a for child: 79bd21cbba42e27ce102925ee9dc168a, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,062 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2011e7580cb04c1da3a10f90f0d5219b for child: e42933689e13bfee2ea375c3661dba49, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,062 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2011e7580cb04c1da3a10f90f0d5219b for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,062 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-3b50b563a593438bad2e04729c60cee6 for child: e42933689e13bfee2ea375c3661dba49, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,062 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/3b50b563a593438bad2e04729c60cee6 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,066 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/411aa2a87b234c4493e1c35e77a09b9a for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,084 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-b908652a136646d986886d9a4fd8c3c6 for child: 79bd21cbba42e27ce102925ee9dc168a, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,085 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/.splits/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-d6f1bde2b0d6406399204149d5e084d9 for child: e42933689e13bfee2ea375c3661dba49, parent: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,085 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/d6f1bde2b0d6406399204149d5e084d9 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,098 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bd1dc076a642c8adbbb4b9971f465c 2023-02-11 09:54:29,115 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,118 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b908652a136646d986886d9a4fd8c3c6 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,124 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,125 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4327659ae8514a3abfbbed93376aa59b 2023-02-11 09:54:29,145 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,148 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,161 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,164 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,164 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,172 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180 for region: a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,172 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region a6cffd29aa384a14f910e924d07d1ce8 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:54:29,208 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 09:54:29,210 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 09:54:29,211 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676109269211"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676109269211"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676109269211"}]},"ts":"1676109269211"} 2023-02-11 09:54:29,211 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269211"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109269211"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109269211"}]},"ts":"1676109269211"} 2023-02-11 09:54:29,211 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269211"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109269211"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109269211"}]},"ts":"1676109269211"} 2023-02-11 09:54:29,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79bd21cbba42e27ce102925ee9dc168a, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e42933689e13bfee2ea375c3661dba49, ASSIGN}] 2023-02-11 09:54:29,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e42933689e13bfee2ea375c3661dba49, ASSIGN 2023-02-11 09:54:29,215 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e42933689e13bfee2ea375c3661dba49, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:54:29,216 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79bd21cbba42e27ce102925ee9dc168a, ASSIGN 2023-02-11 09:54:29,216 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79bd21cbba42e27ce102925ee9dc168a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:54:29,366 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 09:54:29,367 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=79bd21cbba42e27ce102925ee9dc168a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:29,367 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269367"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109269367"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109269367"}]},"ts":"1676109269367"} 2023-02-11 09:54:29,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 79bd21cbba42e27ce102925ee9dc168a, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:29,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e42933689e13bfee2ea375c3661dba49, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:29,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269372"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109269372"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109269372"}]},"ts":"1676109269372"} 2023-02-11 09:54:29,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure e42933689e13bfee2ea375c3661dba49, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:29,479 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:54:29,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 79bd21cbba42e27ce102925ee9dc168a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.', STARTKEY => '', ENDKEY => '60'} 2023-02-11 09:54:29,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:29,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:29,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,525 INFO [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,526 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1 2023-02-11 09:54:29,526 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1 2023-02-11 09:54:29,527 INFO [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79bd21cbba42e27ce102925ee9dc168a columnFamilyName cf1 2023-02-11 09:54:29,535 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,537 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b-bottom 2023-02-11 09:54:29,541 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e-bottom 2023-02-11 09:54:29,545 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180-bottom 2023-02-11 09:54:29,549 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f-bottom 2023-02-11 09:54:29,552 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2f268b53a3da42da966de356108d75a0 2023-02-11 09:54:29,555 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-625f7386e776487aa1388d1d7fb926fd 2023-02-11 09:54:29,559 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-b908652a136646d986886d9a4fd8c3c6 2023-02-11 09:54:29,559 INFO [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(310): Store=79bd21cbba42e27ce102925ee9dc168a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:29,559 INFO [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,560 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2 2023-02-11 09:54:29,560 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2 2023-02-11 09:54:29,561 INFO [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79bd21cbba42e27ce102925ee9dc168a columnFamilyName cf2 2023-02-11 09:54:29,571 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858-bottom 2023-02-11 09:54:29,576 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2-bottom 2023-02-11 09:54:29,581 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f-bottom 2023-02-11 09:54:29,584 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,585 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c-bottom 2023-02-11 09:54:29,588 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-411aa2a87b234c4493e1c35e77a09b9a 2023-02-11 09:54:29,591 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-41e82de59ed5419aa97d5aea99e71a46 2023-02-11 09:54:29,593 DEBUG [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-c7ab338240bf48728e98e93a5227e59a 2023-02-11 09:54:29,594 INFO [StoreOpener-79bd21cbba42e27ce102925ee9dc168a-1] regionserver.HStore(310): Store=79bd21cbba42e27ce102925ee9dc168a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:29,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:29,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:29,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 79bd21cbba42e27ce102925ee9dc168a; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66626734, jitterRate=-0.0071842968463897705}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:29,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 79bd21cbba42e27ce102925ee9dc168a: Running coprocessor pre-open hook at 1676109269524Writing region info on filesystem at 1676109269524Initializing all the Stores at 1676109269525 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109269525Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109269525Cleaning up temporary data from old regions at 1676109269596 (+71 ms)Cleaning up detritus from prior splits at 1676109269596Running coprocessor post-open hooks at 1676109269599 (+3 ms)Region opened successfully at 1676109269599 2023-02-11 09:54:29,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a., pid=77, masterSystemTime=1676109269521 2023-02-11 09:54:29,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:29,601 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:29,601 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:29,601 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,601 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1912): 79bd21cbba42e27ce102925ee9dc168a/cf2 is initiating minor compaction (all files) 2023-02-11 09:54:29,601 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,601 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 79bd21cbba42e27ce102925ee9dc168a/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,601 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1912): 79bd21cbba42e27ce102925ee9dc168a/cf1 is initiating minor compaction (all files) 2023-02-11 09:54:29,602 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 79bd21cbba42e27ce102925ee9dc168a/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,602 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-41e82de59ed5419aa97d5aea99e71a46, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-411aa2a87b234c4493e1c35e77a09b9a, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-c7ab338240bf48728e98e93a5227e59a, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp, totalSize=35.0 K 2023-02-11 09:54:29,602 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-625f7386e776487aa1388d1d7fb926fd, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2f268b53a3da42da966de356108d75a0, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-b908652a136646d986886d9a4fd8c3c6, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp, totalSize=35.0 K 2023-02-11 09:54:29,602 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109267076 2023-02-11 09:54:29,603 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109267076 2023-02-11 09:54:29,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,603 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109267268 2023-02-11 09:54:29,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e42933689e13bfee2ea375c3661dba49, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.', STARTKEY => '60', ENDKEY => ''} 2023-02-11 09:54:29,603 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109267268 2023-02-11 09:54:29,603 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=79bd21cbba42e27ce102925ee9dc168a, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:29,603 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109267470 2023-02-11 09:54:29,603 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269603"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109269603"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109269603"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109269603"}]},"ts":"1676109269603"} 2023-02-11 09:54:29,603 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109267470 2023-02-11 09:54:29,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:29,604 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-41e82de59ed5419aa97d5aea99e71a46, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109267673 2023-02-11 09:54:29,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:29,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,604 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-625f7386e776487aa1388d1d7fb926fd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109267673 2023-02-11 09:54:29,604 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-411aa2a87b234c4493e1c35e77a09b9a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109267860 2023-02-11 09:54:29,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,604 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2f268b53a3da42da966de356108d75a0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109267860 2023-02-11 09:54:29,605 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-b908652a136646d986886d9a4fd8c3c6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109268040 2023-02-11 09:54:29,605 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-c7ab338240bf48728e98e93a5227e59a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109268040 2023-02-11 09:54:29,605 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109268220 2023-02-11 09:54:29,606 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109268220 2023-02-11 09:54:29,606 INFO [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-11 09:54:29,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 79bd21cbba42e27ce102925ee9dc168a, server=jenkins-hbase4.apache.org,40095,1676109241701 in 236 msec 2023-02-11 09:54:29,608 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1 2023-02-11 09:54:29,608 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1 2023-02-11 09:54:29,608 INFO [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e42933689e13bfee2ea375c3661dba49 columnFamilyName cf1 2023-02-11 09:54:29,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79bd21cbba42e27ce102925ee9dc168a, ASSIGN in 394 msec 2023-02-11 09:54:29,618 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,618 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b-top 2023-02-11 09:54:29,620 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79bd21cbba42e27ce102925ee9dc168a#cf1#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,620 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] throttle.PressureAwareThroughputController(145): 79bd21cbba42e27ce102925ee9dc168a#cf2#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,622 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e-top 2023-02-11 09:54:29,625 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180-top 2023-02-11 09:54:29,627 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f-top 2023-02-11 09:54:29,634 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:29,634 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:29,638 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:29,638 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:29,641 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp/cf2/ea60193750e145979e2a4390ae86c581 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/ea60193750e145979e2a4390ae86c581 2023-02-11 09:54:29,642 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:29,644 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:29,644 INFO [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(310): Store=e42933689e13bfee2ea375c3661dba49/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:29,644 INFO [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,644 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp/cf1/91d5f4e82fe048688ed10e0d37aa6ac7 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/91d5f4e82fe048688ed10e0d37aa6ac7 2023-02-11 09:54:29,645 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2 2023-02-11 09:54:29,645 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2 2023-02-11 09:54:29,645 INFO [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e42933689e13bfee2ea375c3661dba49 columnFamilyName cf2 2023-02-11 09:54:29,647 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 79bd21cbba42e27ce102925ee9dc168a/cf2 of 79bd21cbba42e27ce102925ee9dc168a into ea60193750e145979e2a4390ae86c581(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,647 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79bd21cbba42e27ce102925ee9dc168a: 2023-02-11 09:54:29,647 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a., storeName=79bd21cbba42e27ce102925ee9dc168a/cf2, priority=9, startTime=1676109269600; duration=0sec 2023-02-11 09:54:29,647 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,649 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 79bd21cbba42e27ce102925ee9dc168a/cf1 of 79bd21cbba42e27ce102925ee9dc168a into 91d5f4e82fe048688ed10e0d37aa6ac7(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,650 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79bd21cbba42e27ce102925ee9dc168a: 2023-02-11 09:54:29,650 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a., storeName=79bd21cbba42e27ce102925ee9dc168a/cf1, priority=9, startTime=1676109269600; duration=0sec 2023-02-11 09:54:29,650 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,661 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858-top 2023-02-11 09:54:29,664 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2-top 2023-02-11 09:54:29,667 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f-top 2023-02-11 09:54:29,670 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:29,670 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c-top 2023-02-11 09:54:29,673 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:29,673 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:29,676 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:29,676 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:29,679 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e034bbe3a64b456797baac442725082f 2023-02-11 09:54:29,679 DEBUG [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-e034bbe3a64b456797baac442725082f 2023-02-11 09:54:29,679 INFO [StoreOpener-e42933689e13bfee2ea375c3661dba49-1] regionserver.HStore(310): Store=e42933689e13bfee2ea375c3661dba49/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:29,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:54:29,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:29,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e42933689e13bfee2ea375c3661dba49; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60699057, jitterRate=-0.09551356732845306}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:54:29,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e42933689e13bfee2ea375c3661dba49: Running coprocessor pre-open hook at 1676109269604Writing region info on filesystem at 1676109269604Initializing all the Stores at 1676109269605 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109269605Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109269605Cleaning up temporary data from old regions at 1676109269681 (+76 ms)Cleaning up detritus from prior splits at 1676109269682 (+1 ms)Running coprocessor post-open hooks at 1676109269683 (+1 ms)Region opened successfully at 1676109269683 2023-02-11 09:54:29,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49., pid=78, masterSystemTime=1676109269521 2023-02-11 09:54:29,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:29,685 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:29,685 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:54:29,685 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,685 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1912): e42933689e13bfee2ea375c3661dba49/cf1 is initiating minor compaction (all files) 2023-02-11 09:54:29,685 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,685 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e42933689e13bfee2ea375c3661dba49/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,685 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1912): e42933689e13bfee2ea375c3661dba49/cf2 is initiating minor compaction (all files) 2023-02-11 09:54:29,686 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e42933689e13bfee2ea375c3661dba49/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,686 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-88a7ccb96c1240edbde24c468cbdcdd3, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2011e7580cb04c1da3a10f90f0d5219b, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-d6f1bde2b0d6406399204149d5e084d9] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp, totalSize=34.8 K 2023-02-11 09:54:29,686 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-126ebc0a0ed640f7886b8e7ea34cc0b1, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-e034bbe3a64b456797baac442725082f, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-3b50b563a593438bad2e04729c60cee6] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp, totalSize=34.8 K 2023-02-11 09:54:29,686 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109267076 2023-02-11 09:54:29,686 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109267076 2023-02-11 09:54:29,687 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109267268 2023-02-11 09:54:29,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,687 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109267268 2023-02-11 09:54:29,687 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109267470 2023-02-11 09:54:29,687 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109267470 2023-02-11 09:54:29,687 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676109268220 2023-02-11 09:54:29,688 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676109268220 2023-02-11 09:54:29,688 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-88a7ccb96c1240edbde24c468cbdcdd3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,688 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-126ebc0a0ed640f7886b8e7ea34cc0b1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,688 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2011e7580cb04c1da3a10f90f0d5219b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,688 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-e034bbe3a64b456797baac442725082f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,688 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-d6f1bde2b0d6406399204149d5e084d9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,689 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e42933689e13bfee2ea375c3661dba49, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:29,689 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-3b50b563a593438bad2e04729c60cee6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,690 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269689"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109269689"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109269689"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109269689"}]},"ts":"1676109269689"} 2023-02-11 09:54:29,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-11 09:54:29,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure e42933689e13bfee2ea375c3661dba49, server=jenkins-hbase4.apache.org,40095,1676109241701 in 318 msec 2023-02-11 09:54:29,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-11 09:54:29,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e42933689e13bfee2ea375c3661dba49, ASSIGN in 479 msec 2023-02-11 09:54:29,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 in 950 msec 2023-02-11 09:54:29,705 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] throttle.PressureAwareThroughputController(145): e42933689e13bfee2ea375c3661dba49#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,705 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] throttle.PressureAwareThroughputController(145): e42933689e13bfee2ea375c3661dba49#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,720 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fa9ad972860a4df58568e88e0d213830: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,720 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 959eaa79a11a402397b349b9485d2a18: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,720 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp/cf2/fa9ad972860a4df58568e88e0d213830 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/fa9ad972860a4df58568e88e0d213830 2023-02-11 09:54:29,721 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp/cf1/959eaa79a11a402397b349b9485d2a18 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/959eaa79a11a402397b349b9485d2a18 2023-02-11 09:54:29,724 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fa9ad972860a4df58568e88e0d213830: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,724 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 959eaa79a11a402397b349b9485d2a18: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,725 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e42933689e13bfee2ea375c3661dba49/cf2 of e42933689e13bfee2ea375c3661dba49 into fa9ad972860a4df58568e88e0d213830(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,725 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e42933689e13bfee2ea375c3661dba49/cf1 of e42933689e13bfee2ea375c3661dba49 into 959eaa79a11a402397b349b9485d2a18(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,725 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e42933689e13bfee2ea375c3661dba49: 2023-02-11 09:54:29,725 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e42933689e13bfee2ea375c3661dba49: 2023-02-11 09:54:29,725 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49., storeName=e42933689e13bfee2ea375c3661dba49/cf2, priority=9, startTime=1676109269685; duration=0sec 2023-02-11 09:54:29,725 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49., storeName=e42933689e13bfee2ea375c3661dba49/cf1, priority=9, startTime=1676109269684; duration=0sec 2023-02-11 09:54:29,725 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,725 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,756 DEBUG [Listener at localhost/34597] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 09:54:29,759 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35044, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 09:54:29,759 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 09:54:29,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] regionserver.HStore(1912): 79bd21cbba42e27ce102925ee9dc168a/cf1 is initiating major compaction (all files) 2023-02-11 09:54:29,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6057310e; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,760 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 79bd21cbba42e27ce102925ee9dc168a/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 09:54:29,760 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/91d5f4e82fe048688ed10e0d37aa6ac7] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp, totalSize=6.8 K 2023-02-11 09:54:29,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] regionserver.HStore(1912): 79bd21cbba42e27ce102925ee9dc168a/cf2 is initiating major compaction (all files) 2023-02-11 09:54:29,761 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@74029d5b; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:29,761 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 79bd21cbba42e27ce102925ee9dc168a/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:29,761 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/ea60193750e145979e2a4390ae86c581] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp, totalSize=6.8 K 2023-02-11 09:54:29,761 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 91d5f4e82fe048688ed10e0d37aa6ac7, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109267076 2023-02-11 09:54:29,761 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting ea60193750e145979e2a4390ae86c581, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109267076 2023-02-11 09:54:29,762 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 09:54:29,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] regionserver.HStore(1912): e42933689e13bfee2ea375c3661dba49/cf1 is initiating major compaction (all files) 2023-02-11 09:54:29,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@31eec3d7; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:54:29,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 09:54:29,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] regionserver.HStore(1912): e42933689e13bfee2ea375c3661dba49/cf2 is initiating major compaction (all files) 2023-02-11 09:54:29,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@35c3cba6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 09:54:29,764 INFO [Listener at localhost/34597] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-11 09:54:29,766 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79bd21cbba42e27ce102925ee9dc168a#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,766 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] throttle.PressureAwareThroughputController(145): 79bd21cbba42e27ce102925ee9dc168a#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,779 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. is not mergeable because it has references 2023-02-11 09:54:29,780 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. is not mergeable because it has references 2023-02-11 09:54:29,783 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp/cf1/a087c5a831a64f9ebad25d643db02a97 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/a087c5a831a64f9ebad25d643db02a97 2023-02-11 09:54:29,783 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/.tmp/cf2/5fe8b55aafaf4998ae148c2a55aa5a0b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5fe8b55aafaf4998ae148c2a55aa5a0b 2023-02-11 09:54:29,787 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 79bd21cbba42e27ce102925ee9dc168a/cf2 of 79bd21cbba42e27ce102925ee9dc168a into 5fe8b55aafaf4998ae148c2a55aa5a0b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,787 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 79bd21cbba42e27ce102925ee9dc168a/cf1 of 79bd21cbba42e27ce102925ee9dc168a into a087c5a831a64f9ebad25d643db02a97(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,787 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79bd21cbba42e27ce102925ee9dc168a: 2023-02-11 09:54:29,787 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79bd21cbba42e27ce102925ee9dc168a: 2023-02-11 09:54:29,789 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676109269760; duration=0sec 2023-02-11 09:54:29,789 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676109269760; duration=0sec 2023-02-11 09:54:29,789 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 09:54:29,789 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 09:54:29,789 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e42933689e13bfee2ea375c3661dba49/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,789 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e42933689e13bfee2ea375c3661dba49/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:29,789 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/fa9ad972860a4df58568e88e0d213830] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp, totalSize=5.0 K 2023-02-11 09:54:29,789 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/959eaa79a11a402397b349b9485d2a18] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp, totalSize=5.0 K 2023-02-11 09:54:29,789 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting fa9ad972860a4df58568e88e0d213830, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,789 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 959eaa79a11a402397b349b9485d2a18, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 09:54:29,792 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] throttle.PressureAwareThroughputController(145): e42933689e13bfee2ea375c3661dba49#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,792 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] throttle.PressureAwareThroughputController(145): e42933689e13bfee2ea375c3661dba49#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:54:29,801 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5b388bb4d3064fa59801705d9f6fd1ca: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,801 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6dc1fe04781d43a99536e91005ef78e5: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,802 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp/cf2/5b388bb4d3064fa59801705d9f6fd1ca as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5b388bb4d3064fa59801705d9f6fd1ca 2023-02-11 09:54:29,802 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/.tmp/cf1/6dc1fe04781d43a99536e91005ef78e5 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/6dc1fe04781d43a99536e91005ef78e5 2023-02-11 09:54:29,805 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5b388bb4d3064fa59801705d9f6fd1ca: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,805 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6dc1fe04781d43a99536e91005ef78e5: NONE, but ROW specified in column family configuration 2023-02-11 09:54:29,806 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e42933689e13bfee2ea375c3661dba49/cf2 of e42933689e13bfee2ea375c3661dba49 into 5b388bb4d3064fa59801705d9f6fd1ca(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,806 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e42933689e13bfee2ea375c3661dba49: 2023-02-11 09:54:29,806 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676109269763; duration=0sec 2023-02-11 09:54:29,806 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e42933689e13bfee2ea375c3661dba49/cf1 of e42933689e13bfee2ea375c3661dba49 into 6dc1fe04781d43a99536e91005ef78e5(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:54:29,806 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,806 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e42933689e13bfee2ea375c3661dba49: 2023-02-11 09:54:29,806 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676109269763; duration=0sec 2023-02-11 09:54:29,806 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:54:29,887 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40095] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. is not mergeable because it has references 2023-02-11 09:54:29,888 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40095] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. is not mergeable because it has references 2023-02-11 09:54:29,894 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-11 09:54:29,896 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:29,897 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:29,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:29,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:29,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 09:54:29,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109269904"}]},"ts":"1676109269904"} 2023-02-11 09:54:29,906 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-11 09:54:29,908 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-11 09:54:29,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79bd21cbba42e27ce102925ee9dc168a, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e42933689e13bfee2ea375c3661dba49, UNASSIGN}] 2023-02-11 09:54:29,910 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e42933689e13bfee2ea375c3661dba49, UNASSIGN 2023-02-11 09:54:29,910 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e42933689e13bfee2ea375c3661dba49, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:29,911 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109269910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109269910"}]},"ts":"1676109269910"} 2023-02-11 09:54:29,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure e42933689e13bfee2ea375c3661dba49, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:29,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79bd21cbba42e27ce102925ee9dc168a, UNASSIGN 2023-02-11 09:54:29,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=79bd21cbba42e27ce102925ee9dc168a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:29,915 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109269915"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109269915"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109269915"}]},"ts":"1676109269915"} 2023-02-11 09:54:29,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 79bd21cbba42e27ce102925ee9dc168a, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:30,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 09:54:30,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:30,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e42933689e13bfee2ea375c3661dba49, disabling compactions & flushes 2023-02-11 09:54:30,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:30,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:30,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. after waiting 0 ms 2023-02-11 09:54:30,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:30,065 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-88a7ccb96c1240edbde24c468cbdcdd3, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2011e7580cb04c1da3a10f90f0d5219b, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/959eaa79a11a402397b349b9485d2a18, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-d6f1bde2b0d6406399204149d5e084d9] to archive 2023-02-11 09:54:30,065 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:30,067 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,068 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,069 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,070 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,071 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-88a7ccb96c1240edbde24c468cbdcdd3 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:30,071 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2011e7580cb04c1da3a10f90f0d5219b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:30,072 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/959eaa79a11a402397b349b9485d2a18 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/959eaa79a11a402397b349b9485d2a18 2023-02-11 09:54:30,073 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-d6f1bde2b0d6406399204149d5e084d9 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:30,074 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-126ebc0a0ed640f7886b8e7ea34cc0b1, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-e034bbe3a64b456797baac442725082f, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/fa9ad972860a4df58568e88e0d213830, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-3b50b563a593438bad2e04729c60cee6] to archive 2023-02-11 09:54:30,075 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:30,076 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,077 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,078 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,079 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,080 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-126ebc0a0ed640f7886b8e7ea34cc0b1 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:30,081 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-e034bbe3a64b456797baac442725082f to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-e034bbe3a64b456797baac442725082f 2023-02-11 09:54:30,082 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/fa9ad972860a4df58568e88e0d213830 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/fa9ad972860a4df58568e88e0d213830 2023-02-11 09:54:30,083 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-3b50b563a593438bad2e04729c60cee6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:30,086 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 09:54:30,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:30,087 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49. 2023-02-11 09:54:30,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e42933689e13bfee2ea375c3661dba49: Waiting for close lock at 1676109270064Running coprocessor pre-close hooks at 1676109270064Disabling compacts and flushes for region at 1676109270064Disabling writes for close at 1676109270064Writing region close event to WAL at 1676109270084 (+20 ms)Running coprocessor post-close hooks at 1676109270087 (+3 ms)Closed at 1676109270087 2023-02-11 09:54:30,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:30,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:30,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 79bd21cbba42e27ce102925ee9dc168a, disabling compactions & flushes 2023-02-11 09:54:30,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:30,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:30,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. after waiting 0 ms 2023-02-11 09:54:30,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:30,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e42933689e13bfee2ea375c3661dba49, regionState=CLOSED 2023-02-11 09:54:30,089 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109270088"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109270088"}]},"ts":"1676109270088"} 2023-02-11 09:54:30,089 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-625f7386e776487aa1388d1d7fb926fd, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2f268b53a3da42da966de356108d75a0, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-b908652a136646d986886d9a4fd8c3c6, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/91d5f4e82fe048688ed10e0d37aa6ac7, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b-bottom] to archive 2023-02-11 09:54:30,090 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:30,091 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/bd79b9e52b6e49949176a4acd45db180.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-11 09:54:30,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure e42933689e13bfee2ea375c3661dba49, server=jenkins-hbase4.apache.org,40095,1676109241701 in 178 msec 2023-02-11 09:54:30,093 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/fe5da30219684eb88f983578dde96f9f.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e42933689e13bfee2ea375c3661dba49, UNASSIGN in 183 msec 2023-02-11 09:54:30,094 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/b47de98067034be1b5dd09cd10e3879e.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,094 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-625f7386e776487aa1388d1d7fb926fd to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-625f7386e776487aa1388d1d7fb926fd 2023-02-11 09:54:30,095 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2f268b53a3da42da966de356108d75a0 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-2f268b53a3da42da966de356108d75a0 2023-02-11 09:54:30,096 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-b908652a136646d986886d9a4fd8c3c6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-b908652a136646d986886d9a4fd8c3c6 2023-02-11 09:54:30,097 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/91d5f4e82fe048688ed10e0d37aa6ac7 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/91d5f4e82fe048688ed10e0d37aa6ac7 2023-02-11 09:54:30,098 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/4327659ae8514a3abfbbed93376aa59b.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,099 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-41e82de59ed5419aa97d5aea99e71a46, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-411aa2a87b234c4493e1c35e77a09b9a, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-c7ab338240bf48728e98e93a5227e59a, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/ea60193750e145979e2a4390ae86c581, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c-bottom] to archive 2023-02-11 09:54:30,100 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:54:30,101 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5f208f1ba251438793cb590f70677ef2.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,102 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/99d47d4616e343dd993b370bc8306f7f.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,103 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/0f1d224c52244d868e8fc16eae295858.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,104 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-41e82de59ed5419aa97d5aea99e71a46 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-41e82de59ed5419aa97d5aea99e71a46 2023-02-11 09:54:30,105 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-411aa2a87b234c4493e1c35e77a09b9a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-411aa2a87b234c4493e1c35e77a09b9a 2023-02-11 09:54:30,106 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-c7ab338240bf48728e98e93a5227e59a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/testSplitTableRegionDeletedRowsDaughter=a6cffd29aa384a14f910e924d07d1ce8-c7ab338240bf48728e98e93a5227e59a 2023-02-11 09:54:30,107 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/ea60193750e145979e2a4390ae86c581 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/ea60193750e145979e2a4390ae86c581 2023-02-11 09:54:30,108 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/d3bd1dc076a642c8adbbb4b9971f465c.a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 09:54:30,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:30,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a. 2023-02-11 09:54:30,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 79bd21cbba42e27ce102925ee9dc168a: Waiting for close lock at 1676109270088Running coprocessor pre-close hooks at 1676109270088Disabling compacts and flushes for region at 1676109270088Disabling writes for close at 1676109270088Writing region close event to WAL at 1676109270109 (+21 ms)Running coprocessor post-close hooks at 1676109270111 (+2 ms)Closed at 1676109270111 2023-02-11 09:54:30,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:30,113 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=79bd21cbba42e27ce102925ee9dc168a, regionState=CLOSED 2023-02-11 09:54:30,113 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676109270113"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109270113"}]},"ts":"1676109270113"} 2023-02-11 09:54:30,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-11 09:54:30,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 79bd21cbba42e27ce102925ee9dc168a, server=jenkins-hbase4.apache.org,40095,1676109241701 in 197 msec 2023-02-11 09:54:30,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-11 09:54:30,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79bd21cbba42e27ce102925ee9dc168a, UNASSIGN in 206 msec 2023-02-11 09:54:30,117 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109270117"}]},"ts":"1676109270117"} 2023-02-11 09:54:30,118 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-11 09:54:30,121 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-11 09:54:30,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 224 msec 2023-02-11 09:54:30,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 09:54:30,203 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-11 09:54:30,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:30,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:30,206 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:30,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 09:54:30,207 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:30,211 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,211 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:30,211 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:30,213 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/recovered.edits] 2023-02-11 09:54:30,213 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/recovered.edits] 2023-02-11 09:54:30,214 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/recovered.edits] 2023-02-11 09:54:30,216 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/6dc1fe04781d43a99536e91005ef78e5 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf1/6dc1fe04781d43a99536e91005ef78e5 2023-02-11 09:54:30,217 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2011e7580cb04c1da3a10f90f0d5219b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2011e7580cb04c1da3a10f90f0d5219b 2023-02-11 09:54:30,217 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/a087c5a831a64f9ebad25d643db02a97 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf1/a087c5a831a64f9ebad25d643db02a97 2023-02-11 09:54:30,218 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2f268b53a3da42da966de356108d75a0 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/2f268b53a3da42da966de356108d75a0 2023-02-11 09:54:30,219 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5b388bb4d3064fa59801705d9f6fd1ca to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/cf2/5b388bb4d3064fa59801705d9f6fd1ca 2023-02-11 09:54:30,220 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5fe8b55aafaf4998ae148c2a55aa5a0b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/cf2/5fe8b55aafaf4998ae148c2a55aa5a0b 2023-02-11 09:54:30,220 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/4327659ae8514a3abfbbed93376aa59b 2023-02-11 09:54:30,222 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/625f7386e776487aa1388d1d7fb926fd to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/625f7386e776487aa1388d1d7fb926fd 2023-02-11 09:54:30,223 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/recovered.edits/113.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49/recovered.edits/113.seqid 2023-02-11 09:54:30,223 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/recovered.edits/113.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a/recovered.edits/113.seqid 2023-02-11 09:54:30,223 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/88a7ccb96c1240edbde24c468cbdcdd3 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/88a7ccb96c1240edbde24c468cbdcdd3 2023-02-11 09:54:30,223 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e42933689e13bfee2ea375c3661dba49 2023-02-11 09:54:30,223 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79bd21cbba42e27ce102925ee9dc168a 2023-02-11 09:54:30,224 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b47de98067034be1b5dd09cd10e3879e 2023-02-11 09:54:30,225 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b908652a136646d986886d9a4fd8c3c6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/b908652a136646d986886d9a4fd8c3c6 2023-02-11 09:54:30,227 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/bd79b9e52b6e49949176a4acd45db180 2023-02-11 09:54:30,228 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/d6f1bde2b0d6406399204149d5e084d9 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/d6f1bde2b0d6406399204149d5e084d9 2023-02-11 09:54:30,229 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf1/fe5da30219684eb88f983578dde96f9f 2023-02-11 09:54:30,231 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/0f1d224c52244d868e8fc16eae295858 2023-02-11 09:54:30,232 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/126ebc0a0ed640f7886b8e7ea34cc0b1 2023-02-11 09:54:30,233 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/3b50b563a593438bad2e04729c60cee6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/3b50b563a593438bad2e04729c60cee6 2023-02-11 09:54:30,234 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/411aa2a87b234c4493e1c35e77a09b9a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/411aa2a87b234c4493e1c35e77a09b9a 2023-02-11 09:54:30,235 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/41e82de59ed5419aa97d5aea99e71a46 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/41e82de59ed5419aa97d5aea99e71a46 2023-02-11 09:54:30,236 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/5f208f1ba251438793cb590f70677ef2 2023-02-11 09:54:30,237 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/99d47d4616e343dd993b370bc8306f7f 2023-02-11 09:54:30,238 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/c7ab338240bf48728e98e93a5227e59a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/c7ab338240bf48728e98e93a5227e59a 2023-02-11 09:54:30,239 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/d3bd1dc076a642c8adbbb4b9971f465c 2023-02-11 09:54:30,240 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/e034bbe3a64b456797baac442725082f to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/cf2/e034bbe3a64b456797baac442725082f 2023-02-11 09:54:30,242 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/recovered.edits/106.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8/recovered.edits/106.seqid 2023-02-11 09:54:30,243 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a6cffd29aa384a14f910e924d07d1ce8 2023-02-11 09:54:30,243 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 09:54:30,245 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:30,247 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-11 09:54:30,248 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-11 09:54:30,249 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:30,249 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-11 09:54:30,249 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109270249"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:30,249 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109270249"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:30,250 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109270249"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:30,251 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 09:54:30,251 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a6cffd29aa384a14f910e924d07d1ce8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676109266573.a6cffd29aa384a14f910e924d07d1ce8.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 79bd21cbba42e27ce102925ee9dc168a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676109268745.79bd21cbba42e27ce102925ee9dc168a.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => e42933689e13bfee2ea375c3661dba49, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676109268745.e42933689e13bfee2ea375c3661dba49.', STARTKEY => '60', ENDKEY => ''}] 2023-02-11 09:54:30,251 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-11 09:54:30,251 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109270251"}]},"ts":"9223372036854775807"} 2023-02-11 09:54:30,252 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-11 09:54:30,254 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:54:30,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 50 msec 2023-02-11 09:54:30,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 09:54:30,308 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-11 09:54:30,323 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=412 (was 406) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:54906 [Waiting for operation #45] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:53934 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:40095-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:56972 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:57754 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:53808 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:45370 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:53826 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:56560 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:45398 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:56328 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:54576 [Waiting for operation #53] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:33010 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:54170 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:56954 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:56298 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-632512390_15 at /127.0.0.1:56536 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 856), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=297 (was 271) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=2724 (was 2742) 2023-02-11 09:54:30,337 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=412, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=297, ProcessCount=175, AvailableMemoryMB=2724 2023-02-11 09:54:30,338 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:54:30,340 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:54:30,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:54:30,342 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:54:30,343 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:54:30,344 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:54:30,346 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,346 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 empty. 2023-02-11 09:54:30,346 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 09:54:30,357 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:30,358 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1138c5b51866ce5ac5da9db13a373fc5, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:54:30,366 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:30,366 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 1138c5b51866ce5ac5da9db13a373fc5, disabling compactions & flushes 2023-02-11 09:54:30,366 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:30,366 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:30,366 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. after waiting 0 ms 2023-02-11 09:54:30,366 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:30,366 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:30,366 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 1138c5b51866ce5ac5da9db13a373fc5: Waiting for close lock at 1676109270366Disabling compacts and flushes for region at 1676109270366Disabling writes for close at 1676109270366Writing region close event to WAL at 1676109270366Closed at 1676109270366 2023-02-11 09:54:30,368 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:54:30,369 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109270369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109270369"}]},"ts":"1676109270369"} 2023-02-11 09:54:30,370 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:54:30,371 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:54:30,371 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109270371"}]},"ts":"1676109270371"} 2023-02-11 09:54:30,372 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-11 09:54:30,376 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:30,376 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:30,376 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:30,376 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:30,376 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:54:30,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, ASSIGN}] 2023-02-11 09:54:30,378 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, ASSIGN 2023-02-11 09:54:30,378 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=false 2023-02-11 09:54:30,529 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:30,531 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:30,531 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109270531"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109270531"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109270531"}]},"ts":"1676109270531"} 2023-02-11 09:54:30,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:30,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:30,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1138c5b51866ce5ac5da9db13a373fc5, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:30,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:30,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:30,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,689 INFO [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,690 DEBUG [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/cf1 2023-02-11 09:54:30,690 DEBUG [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/cf1 2023-02-11 09:54:30,691 INFO [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1138c5b51866ce5ac5da9db13a373fc5 columnFamilyName cf1 2023-02-11 09:54:30,691 INFO [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] regionserver.HStore(310): Store=1138c5b51866ce5ac5da9db13a373fc5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:30,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:30,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:54:30,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1138c5b51866ce5ac5da9db13a373fc5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73369748, jitterRate=0.09329444169998169}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 09:54:30,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1138c5b51866ce5ac5da9db13a373fc5: Running coprocessor pre-open hook at 1676109270688Writing region info on filesystem at 1676109270688Initializing all the Stores at 1676109270688Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109270688Cleaning up temporary data from old regions at 1676109270693 (+5 ms)Cleaning up detritus from prior splits at 1676109270693Running coprocessor post-open hooks at 1676109270697 (+4 ms)Region opened successfully at 1676109270697 2023-02-11 09:54:30,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5., pid=87, masterSystemTime=1676109270684 2023-02-11 09:54:30,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:30,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:30,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:30,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109270699"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109270699"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109270699"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109270699"}]},"ts":"1676109270699"} 2023-02-11 09:54:30,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-11 09:54:30,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745 in 168 msec 2023-02-11 09:54:30,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-11 09:54:30,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, ASSIGN in 325 msec 2023-02-11 09:54:30,704 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:54:30,704 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109270704"}]},"ts":"1676109270704"} 2023-02-11 09:54:30,705 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-11 09:54:30,712 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:54:30,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 371 msec 2023-02-11 09:54:30,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 09:54:30,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:54:30,870 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 09:54:30,871 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 09:54:30,871 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 09:54:30,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-11 09:54:30,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, REOPEN/MOVE}] 2023-02-11 09:54:30,888 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, REOPEN/MOVE 2023-02-11 09:54:30,889 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:30,889 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109270889"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109270889"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109270889"}]},"ts":"1676109270889"} 2023-02-11 09:54:30,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:31,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1138c5b51866ce5ac5da9db13a373fc5, disabling compactions & flushes 2023-02-11 09:54:31,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. after waiting 0 ms 2023-02-11 09:54:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 09:54:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:54:31,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1138c5b51866ce5ac5da9db13a373fc5: Waiting for close lock at 1676109271043Running coprocessor pre-close hooks at 1676109271043Disabling compacts and flushes for region at 1676109271043Disabling writes for close at 1676109271044 (+1 ms)Writing region close event to WAL at 1676109271044Running coprocessor post-close hooks at 1676109271047 (+3 ms)Closed at 1676109271047 2023-02-11 09:54:31,047 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 1138c5b51866ce5ac5da9db13a373fc5 to self. 2023-02-11 09:54:31,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=CLOSED 2023-02-11 09:54:31,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109271049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109271049"}]},"ts":"1676109271049"} 2023-02-11 09:54:31,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-11 09:54:31,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745 in 160 msec 2023-02-11 09:54:31,052 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34799,1676109241745; forceNewPlan=false, retain=true 2023-02-11 09:54:31,202 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:54:31,203 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:31,203 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109271203"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109271203"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109271203"}]},"ts":"1676109271203"} 2023-02-11 09:54:31,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:54:31,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:31,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1138c5b51866ce5ac5da9db13a373fc5, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:54:31,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:31,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:31,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,360 INFO [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,361 DEBUG [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/cf1 2023-02-11 09:54:31,361 DEBUG [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/cf1 2023-02-11 09:54:31,362 INFO [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1138c5b51866ce5ac5da9db13a373fc5 columnFamilyName cf1 2023-02-11 09:54:31,362 INFO [StoreOpener-1138c5b51866ce5ac5da9db13a373fc5-1] regionserver.HStore(310): Store=1138c5b51866ce5ac5da9db13a373fc5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:54:31,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:54:31,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1138c5b51866ce5ac5da9db13a373fc5; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71924526, jitterRate=0.07175895571708679}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 09:54:31,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1138c5b51866ce5ac5da9db13a373fc5: Running coprocessor pre-open hook at 1676109271359Writing region info on filesystem at 1676109271359Initializing all the Stores at 1676109271360 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109271360Cleaning up temporary data from old regions at 1676109271364 (+4 ms)Cleaning up detritus from prior splits at 1676109271365 (+1 ms)Running coprocessor post-open hooks at 1676109271367 (+2 ms)Region opened successfully at 1676109271367 2023-02-11 09:54:31,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5., pid=92, masterSystemTime=1676109271356 2023-02-11 09:54:31,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:31,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:54:31,370 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:54:31,370 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109271370"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109271370"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109271370"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109271370"}]},"ts":"1676109271370"} 2023-02-11 09:54:31,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-11 09:54:31,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745 in 167 msec 2023-02-11 09:54:31,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-11 09:54:31,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, REOPEN/MOVE in 485 msec 2023-02-11 09:54:31,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-11 09:54:31,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 501 msec 2023-02-11 09:54:31,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77, ASSIGN}] 2023-02-11 09:54:31,381 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77, ASSIGN 2023-02-11 09:54:31,381 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 09:54:31,532 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:54:31,533 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:54:31,533 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:54:31,533 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:54:31,533 DEBUG [jenkins-hbase4:43041] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-11 09:54:31,535 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=6cedf5c3c6e687883cbad285602a2b77, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:54:31,535 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109271534"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676109271534"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676109271534"}]},"ts":"1676109271534"} 2023-02-11 09:54:31,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 6cedf5c3c6e687883cbad285602a2b77, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:54:31,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. 2023-02-11 09:54:31,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6cedf5c3c6e687883cbad285602a2b77, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-11 09:54:31,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:54:31,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 6cedf5c3c6e687883cbad285602a2b77 2023-02-11 09:54:31,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:54:31,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6cedf5c3c6e687883cbad285602a2b77 2023-02-11 09:54:31,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6cedf5c3c6e687883cbad285602a2b77 2023-02-11 09:54:31,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:31,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:31,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:31,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:32,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,043 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-11 09:54:33,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:33,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,446 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:54:34,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:34,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:35,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:36,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:37,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:38,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:39,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:40,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:41,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:42,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:43,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:44,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:45,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:46,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:47,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:48,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:49,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:50,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:51,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:52,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:53,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:54,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:55,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:56,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:57,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:58,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:54:59,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:00,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:01,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:02,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,058 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 09:55:03,080 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 09:55:03,106 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 09:55:03,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:03,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:04,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:05,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:06,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:07,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:08,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:09,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:10,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:11,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:12,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:13,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:14,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:15,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:16,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:17,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:18,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:19,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:20,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:21,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:22,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:23,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:24,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:25,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:26,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:27,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:28,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:29,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:30,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 09:55:31,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 09:55:31,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 09:55:31,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 09:55:31,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:31,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 09:55:32,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:32,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:33,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:33,071 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:33,072 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:33,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:33,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:33,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:33,075 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 1138c5b51866ce5ac5da9db13a373fc5, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:33,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 exec-time=6 msec 2023-02-11 09:55:33,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:33,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:33,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 09:55:33,412 INFO [StoreOpener-6cedf5c3c6e687883cbad285602a2b77-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6cedf5c3c6e687883cbad285602a2b77 2023-02-11 09:55:33,416 DEBUG [StoreOpener-6cedf5c3c6e687883cbad285602a2b77-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/cf1 2023-02-11 09:55:33,416 DEBUG [StoreOpener-6cedf5c3c6e687883cbad285602a2b77-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/cf1 2023-02-11 09:55:33,416 INFO [StoreOpener-6cedf5c3c6e687883cbad285602a2b77-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6cedf5c3c6e687883cbad285602a2b77 columnFamilyName cf1 2023-02-11 09:55:33,417 INFO [StoreOpener-6cedf5c3c6e687883cbad285602a2b77-1] regionserver.HStore(310): Store=6cedf5c3c6e687883cbad285602a2b77/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:33,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6cedf5c3c6e687883cbad285602a2b77; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58850479, jitterRate=-0.12305952608585358}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 09:55:33,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6cedf5c3c6e687883cbad285602a2b77: Running coprocessor pre-open hook at 1676109271692Initializing all the Stores at 1676109333412 (+61720 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109333412Running coprocessor post-open hooks at 1676109333418 (+6 ms)Region opened successfully at 1676109333418 2023-02-11 09:55:33,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77., pid=94, masterSystemTime=1676109271689 2023-02-11 09:55:33,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. 2023-02-11 09:55:33,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. 2023-02-11 09:55:33,420 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=6cedf5c3c6e687883cbad285602a2b77, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:33,420 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109333420"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676109333420"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676109333420"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676109333420"}]},"ts":"1676109333420"} 2023-02-11 09:55:33,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-11 09:55:33,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 6cedf5c3c6e687883cbad285602a2b77, server=jenkins-hbase4.apache.org,40095,1676109241701 in 1 mins, 1.885 sec 2023-02-11 09:55:33,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-11 09:55:33,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77, ASSIGN in 1 mins, 2.043 sec 2023-02-11 09:55:33,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.572 sec 2023-02-11 09:55:33,428 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676109270848","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62579,"client":"172.31.14.131:52560","queuetimems":0,"class":"HMaster"} 2023-02-11 09:55:33,428 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-11 09:55:35,323 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up 2023-02-11 09:55:35,325 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:35,326 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 09:55:35,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109335330"}]},"ts":"1676109335330"} 2023-02-11 09:55:35,331 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-11 09:55:35,333 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-11 09:55:35,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77, UNASSIGN}] 2023-02-11 09:55:35,334 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77, UNASSIGN 2023-02-11 09:55:35,335 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6cedf5c3c6e687883cbad285602a2b77, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:35,335 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109335335"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676109335335"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676109335335"}]},"ts":"1676109335335"} 2023-02-11 09:55:35,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 6cedf5c3c6e687883cbad285602a2b77, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:35,338 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, UNASSIGN 2023-02-11 09:55:35,338 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:35,338 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109335338"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109335338"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109335338"}]},"ts":"1676109335338"} 2023-02-11 09:55:35,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745}] 2023-02-11 09:55:35,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 09:55:35,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6cedf5c3c6e687883cbad285602a2b77 2023-02-11 09:55:35,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6cedf5c3c6e687883cbad285602a2b77, disabling compactions & flushes 2023-02-11 09:55:35,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. 2023-02-11 09:55:35,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. 2023-02-11 09:55:35,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. after waiting 0 ms 2023-02-11 09:55:35,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. 2023-02-11 09:55:35,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:35,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77. 2023-02-11 09:55:35,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6cedf5c3c6e687883cbad285602a2b77: Waiting for close lock at 1676109335488Running coprocessor pre-close hooks at 1676109335488Disabling compacts and flushes for region at 1676109335488Disabling writes for close at 1676109335489 (+1 ms)Writing region close event to WAL at 1676109335489Running coprocessor post-close hooks at 1676109335489Closed at 1676109335490 (+1 ms) 2023-02-11 09:55:35,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:55:35,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1138c5b51866ce5ac5da9db13a373fc5, disabling compactions & flushes 2023-02-11 09:55:35,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:55:35,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:55:35,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. after waiting 0 ms 2023-02-11 09:55:35,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:55:35,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6cedf5c3c6e687883cbad285602a2b77 2023-02-11 09:55:35,492 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6cedf5c3c6e687883cbad285602a2b77, regionState=CLOSED 2023-02-11 09:55:35,493 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109335492"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676109335492"}]},"ts":"1676109335492"} 2023-02-11 09:55:35,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-11 09:55:35,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 6cedf5c3c6e687883cbad285602a2b77, server=jenkins-hbase4.apache.org,40095,1676109241701 in 159 msec 2023-02-11 09:55:35,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 09:55:35,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:35,497 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5. 2023-02-11 09:55:35,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1138c5b51866ce5ac5da9db13a373fc5: Waiting for close lock at 1676109335491Running coprocessor pre-close hooks at 1676109335491Disabling compacts and flushes for region at 1676109335491Disabling writes for close at 1676109335492 (+1 ms)Writing region close event to WAL at 1676109335492Running coprocessor post-close hooks at 1676109335497 (+5 ms)Closed at 1676109335497 2023-02-11 09:55:35,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77, UNASSIGN in 162 msec 2023-02-11 09:55:35,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:55:35,499 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=1138c5b51866ce5ac5da9db13a373fc5, regionState=CLOSED 2023-02-11 09:55:35,499 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676109335499"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109335499"}]},"ts":"1676109335499"} 2023-02-11 09:55:35,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-11 09:55:35,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 1138c5b51866ce5ac5da9db13a373fc5, server=jenkins-hbase4.apache.org,34799,1676109241745 in 161 msec 2023-02-11 09:55:35,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-11 09:55:35,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1138c5b51866ce5ac5da9db13a373fc5, UNASSIGN in 167 msec 2023-02-11 09:55:35,504 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109335503"}]},"ts":"1676109335503"} 2023-02-11 09:55:35,505 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-11 09:55:35,506 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-11 09:55:35,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 180 msec 2023-02-11 09:55:35,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 09:55:35,631 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-11 09:55:35,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,634 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,635 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 09:55:35,639 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:55:35,640 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/recovered.edits] 2023-02-11 09:55:35,644 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/recovered.edits/7.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5/recovered.edits/7.seqid 2023-02-11 09:55:35,644 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/1138c5b51866ce5ac5da9db13a373fc5 2023-02-11 09:55:35,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 09:55:35,646 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,649 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-11 09:55:35,651 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-11 09:55:35,651 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,652 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-11 09:55:35,652 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109335652"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:35,652 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109335652"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:35,653 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-11 09:55:35,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1138c5b51866ce5ac5da9db13a373fc5, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676109270341.1138c5b51866ce5ac5da9db13a373fc5.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6cedf5c3c6e687883cbad285602a2b77, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676109270341_0001.6cedf5c3c6e687883cbad285602a2b77.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-11 09:55:35,653 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-11 09:55:35,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109335653"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:35,655 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-11 09:55:35,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:35,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 25 msec 2023-02-11 09:55:35,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 09:55:35,736 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-11 09:55:35,748 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=346 (was 412), OpenFileDescriptor=656 (was 855), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=105 (was 297), ProcessCount=173 (was 175), AvailableMemoryMB=2555 (was 2724) 2023-02-11 09:55:35,760 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=347, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=105, ProcessCount=173, AvailableMemoryMB=2555 2023-02-11 09:55:35,761 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:35,763 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:55:35,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:55:35,764 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:35,766 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:55:35,766 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:55:35,768 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:35,768 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43 empty. 2023-02-11 09:55:35,768 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:35,768 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 09:55:35,777 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-11 09:55:35,778 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8aaae3a9f0c9f48611203f0004ccdf43, NAME => 'testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:55:35,786 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:35,786 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 8aaae3a9f0c9f48611203f0004ccdf43, disabling compactions & flushes 2023-02-11 09:55:35,786 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:35,786 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:35,786 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. after waiting 0 ms 2023-02-11 09:55:35,786 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:35,786 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:35,786 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 8aaae3a9f0c9f48611203f0004ccdf43: Waiting for close lock at 1676109335786Disabling compacts and flushes for region at 1676109335786Disabling writes for close at 1676109335786Writing region close event to WAL at 1676109335786Closed at 1676109335786 2023-02-11 09:55:35,787 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:55:35,788 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676109335788"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109335788"}]},"ts":"1676109335788"} 2023-02-11 09:55:35,789 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:55:35,790 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:55:35,790 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109335790"}]},"ts":"1676109335790"} 2023-02-11 09:55:35,791 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-11 09:55:35,794 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:55:35,795 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:55:35,795 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:55:35,795 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:55:35,795 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:55:35,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8aaae3a9f0c9f48611203f0004ccdf43, ASSIGN}] 2023-02-11 09:55:35,796 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8aaae3a9f0c9f48611203f0004ccdf43, ASSIGN 2023-02-11 09:55:35,797 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=8aaae3a9f0c9f48611203f0004ccdf43, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41949,1676109241776; forceNewPlan=false, retain=false 2023-02-11 09:55:35,947 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:55:35,948 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=8aaae3a9f0c9f48611203f0004ccdf43, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:35,948 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676109335948"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109335948"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109335948"}]},"ts":"1676109335948"} 2023-02-11 09:55:35,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 8aaae3a9f0c9f48611203f0004ccdf43, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:36,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8aaae3a9f0c9f48611203f0004ccdf43, NAME => 'testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:55:36,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:36,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:36,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,107 INFO [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,108 DEBUG [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1 2023-02-11 09:55:36,108 DEBUG [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1 2023-02-11 09:55:36,109 INFO [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8aaae3a9f0c9f48611203f0004ccdf43 columnFamilyName cf1 2023-02-11 09:55:36,109 INFO [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] regionserver.HStore(310): Store=8aaae3a9f0c9f48611203f0004ccdf43/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:36,109 INFO [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,110 DEBUG [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2 2023-02-11 09:55:36,110 DEBUG [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2 2023-02-11 09:55:36,111 INFO [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8aaae3a9f0c9f48611203f0004ccdf43 columnFamilyName cf2 2023-02-11 09:55:36,111 INFO [StoreOpener-8aaae3a9f0c9f48611203f0004ccdf43-1] regionserver.HStore(310): Store=8aaae3a9f0c9f48611203f0004ccdf43/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:36,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:36,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:36,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:55:36,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8aaae3a9f0c9f48611203f0004ccdf43; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62014167, jitterRate=-0.07591690123081207}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:36,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8aaae3a9f0c9f48611203f0004ccdf43: Running coprocessor pre-open hook at 1676109336106Writing region info on filesystem at 1676109336106Initializing all the Stores at 1676109336107 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109336107Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109336107Cleaning up temporary data from old regions at 1676109336113 (+6 ms)Cleaning up detritus from prior splits at 1676109336113Running coprocessor post-open hooks at 1676109336117 (+4 ms)Region opened successfully at 1676109336117 2023-02-11 09:55:36,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43., pid=104, masterSystemTime=1676109336101 2023-02-11 09:55:36,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,121 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=8aaae3a9f0c9f48611203f0004ccdf43, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676109336121"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109336121"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109336121"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109336121"}]},"ts":"1676109336121"} 2023-02-11 09:55:36,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-11 09:55:36,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 8aaae3a9f0c9f48611203f0004ccdf43, server=jenkins-hbase4.apache.org,41949,1676109241776 in 173 msec 2023-02-11 09:55:36,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-11 09:55:36,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=8aaae3a9f0c9f48611203f0004ccdf43, ASSIGN in 329 msec 2023-02-11 09:55:36,126 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:55:36,126 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109336126"}]},"ts":"1676109336126"} 2023-02-11 09:55:36,127 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-11 09:55:36,129 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:55:36,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 365 msec 2023-02-11 09:55:36,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:36,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 09:55:36,274 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 09:55:36,274 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,274 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 09:55:36,274 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 09:55:36,274 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,274 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,276 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,276 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,276 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,276 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,276 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,276 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:36,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,277 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,277 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:36,278 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,278 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. started... 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,280 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aaae3a9f0c9f48611203f0004ccdf43 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:55:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,278 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,283 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,283 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,283 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,283 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,284 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,284 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,289 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/453cd95e493941c89e3d44bfc46602f2 2023-02-11 09:55:36,299 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/a75ab0466cea4cfa8fc6a8ea460cab2a 2023-02-11 09:55:36,303 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/453cd95e493941c89e3d44bfc46602f2 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/453cd95e493941c89e3d44bfc46602f2 2023-02-11 09:55:36,306 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/453cd95e493941c89e3d44bfc46602f2, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:36,307 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/a75ab0466cea4cfa8fc6a8ea460cab2a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/a75ab0466cea4cfa8fc6a8ea460cab2a 2023-02-11 09:55:36,311 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/a75ab0466cea4cfa8fc6a8ea460cab2a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:36,311 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8aaae3a9f0c9f48611203f0004ccdf43 in 31ms, sequenceid=6, compaction requested=false 2023-02-11 09:55:36,312 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-11 09:55:36,312 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aaae3a9f0c9f48611203f0004ccdf43: 2023-02-11 09:55:36,312 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,314 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,314 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,314 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,314 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 09:55:36,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3085caa0[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:36,316 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 09:55:36,316 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,317 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,317 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,317 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,318 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,318 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:36,321 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,321 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,321 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,321 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,321 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,321 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,322 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,322 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,322 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,322 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 09:55:36,325 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 09:55:36,325 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:36,325 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:36,325 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 09:55:36,325 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:36,326 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,326 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,326 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,326 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,327 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,326 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,326 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,327 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,327 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,328 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,330 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,330 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,330 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,333 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 09:55:36,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:36,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,333 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:36,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,334 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,335 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,335 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,434 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:36,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:36,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:36,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:36,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 09:55:36,449 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 09:55:36,449 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,449 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 09:55:36,449 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 09:55:36,449 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,450 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,452 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,452 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,452 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,452 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,453 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,453 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:36,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,454 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,454 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,454 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,455 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,455 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. started... 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,455 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,455 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aaae3a9f0c9f48611203f0004ccdf43 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:36,460 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,460 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,460 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,460 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,460 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,460 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,470 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/5fdb0192361141c1955636e64f9a4d46 2023-02-11 09:55:36,480 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/b67ae91f3b9e4ce4b208aaad77d0638d 2023-02-11 09:55:36,484 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/5fdb0192361141c1955636e64f9a4d46 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/5fdb0192361141c1955636e64f9a4d46 2023-02-11 09:55:36,487 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/5fdb0192361141c1955636e64f9a4d46, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:36,488 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/b67ae91f3b9e4ce4b208aaad77d0638d as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/b67ae91f3b9e4ce4b208aaad77d0638d 2023-02-11 09:55:36,492 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/b67ae91f3b9e4ce4b208aaad77d0638d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:36,493 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8aaae3a9f0c9f48611203f0004ccdf43 in 38ms, sequenceid=19, compaction requested=false 2023-02-11 09:55:36,494 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aaae3a9f0c9f48611203f0004ccdf43: 2023-02-11 09:55:36,494 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,494 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:36,494 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:36,494 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,494 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,494 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,495 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,495 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,496 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,496 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 09:55:36,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@205ccd6a[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:36,498 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 09:55:36,498 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,499 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,499 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,499 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,500 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,500 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:36,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:36,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,500 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:36,502 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,502 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,502 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,503 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,503 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,503 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,503 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,503 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,503 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,503 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:36,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 09:55:36,507 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 09:55:36,507 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:36,507 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:36,507 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 09:55:36,507 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:36,508 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,508 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,508 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,510 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,510 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,510 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,512 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,513 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,513 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,515 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,516 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 09:55:36,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:36,515 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,516 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,516 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:36,516 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:36,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,516 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,516 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,516 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,616 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:36,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:36,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:36,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:36,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 09:55:36,630 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 09:55:36,630 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,630 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 09:55:36,630 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 09:55:36,631 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,631 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,633 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,633 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,633 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,633 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,633 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,633 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,633 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,634 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,634 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,634 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,634 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:36,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:36,635 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,635 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. started... 2023-02-11 09:55:36,635 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aaae3a9f0c9f48611203f0004ccdf43 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:36,636 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,636 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,636 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,637 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,637 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,637 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,645 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/c79b078b635a46188506ab4651bccb8a 2023-02-11 09:55:36,658 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/5c4be7c3d810411889fdeb327952dbb9 2023-02-11 09:55:36,661 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/c79b078b635a46188506ab4651bccb8a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/c79b078b635a46188506ab4651bccb8a 2023-02-11 09:55:36,665 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/c79b078b635a46188506ab4651bccb8a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:36,666 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/5c4be7c3d810411889fdeb327952dbb9 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/5c4be7c3d810411889fdeb327952dbb9 2023-02-11 09:55:36,669 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/5c4be7c3d810411889fdeb327952dbb9, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:36,670 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8aaae3a9f0c9f48611203f0004ccdf43 in 35ms, sequenceid=32, compaction requested=true 2023-02-11 09:55:36,670 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aaae3a9f0c9f48611203f0004ccdf43: 2023-02-11 09:55:36,670 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,670 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:36,670 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:36,670 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,670 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,670 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,672 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,672 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,672 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,672 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 09:55:36,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56099310[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:36,674 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 09:55:36,674 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,675 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,675 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:36,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,675 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,676 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,676 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:36,675 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,676 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,676 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:36,676 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,676 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,680 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,680 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:36,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 09:55:36,683 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 09:55:36,683 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:36,683 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:36,683 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 09:55:36,683 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:36,685 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,685 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,685 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,686 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,686 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,688 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,688 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,688 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 09:55:36,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,695 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,694 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,695 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:36,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,694 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,695 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,695 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,695 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,695 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,695 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,696 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,696 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,696 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,795 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:36,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:36,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:36,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:36,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 09:55:36,809 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 09:55:36,809 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,809 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 09:55:36,809 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 09:55:36,809 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,810 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,811 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,811 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,811 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:36,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:36,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,813 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:36,813 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,813 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:36,813 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:36,813 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,813 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. started... 2023-02-11 09:55:36,814 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aaae3a9f0c9f48611203f0004ccdf43 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:36,814 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,815 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,815 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,815 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,815 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,815 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,845 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/86131bfe566f4518b50c6dc236f590da 2023-02-11 09:55:36,856 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/0d7c54a28d7c41d19b853e71e9d8208b 2023-02-11 09:55:36,860 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/86131bfe566f4518b50c6dc236f590da as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/86131bfe566f4518b50c6dc236f590da 2023-02-11 09:55:36,864 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/86131bfe566f4518b50c6dc236f590da, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:36,865 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/0d7c54a28d7c41d19b853e71e9d8208b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/0d7c54a28d7c41d19b853e71e9d8208b 2023-02-11 09:55:36,869 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/0d7c54a28d7c41d19b853e71e9d8208b, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:36,870 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8aaae3a9f0c9f48611203f0004ccdf43 in 55ms, sequenceid=45, compaction requested=true 2023-02-11 09:55:36,870 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aaae3a9f0c9f48611203f0004ccdf43: 2023-02-11 09:55:36,870 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:36,872 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,872 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,873 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,873 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 09:55:36,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60d0b78e[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:36,876 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 09:55:36,876 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,877 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,877 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,877 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,878 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,878 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,878 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:36,878 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,878 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,878 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,878 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:36,878 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,880 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,880 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,881 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:36,881 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,881 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:36,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 09:55:36,886 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 09:55:36,886 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:36,886 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:36,886 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 09:55:36,886 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:36,898 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,898 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,898 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:36,899 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,900 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,900 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:36,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,902 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:36,902 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:36,903 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:36,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 09:55:36,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:36,908 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:36,909 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:36,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:36,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:36,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:36,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:36,910 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:36,910 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:36,910 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:36,910 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:36,910 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:36,910 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:36,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:36,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,009 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:37,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:37,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:37,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:37,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 09:55:37,023 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 09:55:37,023 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,023 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 09:55:37,023 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 09:55:37,024 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,024 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,025 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,025 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,025 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,025 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,026 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,026 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:37,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:37,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:37,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,027 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:37,028 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:37,028 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:37,028 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,028 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. started... 2023-02-11 09:55:37,028 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aaae3a9f0c9f48611203f0004ccdf43 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:37,028 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:37,036 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/1851f053ea5646d382e95217e12e6b92 2023-02-11 09:55:37,046 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/8509eda6e0a24bbc8ff3c4a7aeca0268 2023-02-11 09:55:37,050 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/1851f053ea5646d382e95217e12e6b92 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/1851f053ea5646d382e95217e12e6b92 2023-02-11 09:55:37,053 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/1851f053ea5646d382e95217e12e6b92, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:37,053 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/8509eda6e0a24bbc8ff3c4a7aeca0268 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/8509eda6e0a24bbc8ff3c4a7aeca0268 2023-02-11 09:55:37,056 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/8509eda6e0a24bbc8ff3c4a7aeca0268, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:37,057 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8aaae3a9f0c9f48611203f0004ccdf43 in 29ms, sequenceid=58, compaction requested=true 2023-02-11 09:55:37,057 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aaae3a9f0c9f48611203f0004ccdf43: 2023-02-11 09:55:37,057 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:37,059 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,059 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:37,059 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,059 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 09:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e1ea1fe[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:37,060 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 09:55:37,061 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,061 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,061 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,061 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,062 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,062 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:37,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:37,062 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:37,066 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:37,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:37,069 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:37,069 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:37,069 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:37,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:37,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:37,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:37,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 09:55:37,071 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 09:55:37,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:37,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:37,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 09:55:37,071 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:37,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:37,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,073 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,073 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,074 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,074 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:37,075 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,076 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,076 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 09:55:37,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:37,078 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:37,078 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,079 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,179 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:37,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:37,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:37,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:37,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 09:55:37,192 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 09:55:37,192 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,192 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 09:55:37,192 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 09:55:37,193 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,193 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,198 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,198 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,198 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,198 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,198 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,198 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:37,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,200 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 09:55:37,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:37,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:37,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:37,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:37,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:37,200 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:37,200 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,200 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. started... 2023-02-11 09:55:37,201 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aaae3a9f0c9f48611203f0004ccdf43 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:37,201 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,202 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,202 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:37,202 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,203 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,203 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:37,209 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/91e3f9bb019d4818970529044a6aee8d 2023-02-11 09:55:37,219 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/e0bbc65f74b54faea1223c9858171800 2023-02-11 09:55:37,224 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/91e3f9bb019d4818970529044a6aee8d as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/91e3f9bb019d4818970529044a6aee8d 2023-02-11 09:55:37,227 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/91e3f9bb019d4818970529044a6aee8d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:37,228 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/e0bbc65f74b54faea1223c9858171800 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/e0bbc65f74b54faea1223c9858171800 2023-02-11 09:55:37,232 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/e0bbc65f74b54faea1223c9858171800, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:37,233 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8aaae3a9f0c9f48611203f0004ccdf43 in 32ms, sequenceid=71, compaction requested=true 2023-02-11 09:55:37,233 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aaae3a9f0c9f48611203f0004ccdf43: 2023-02-11 09:55:37,233 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,233 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:37,233 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:37,233 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,233 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 09:55:37,233 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 09:55:37,235 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,235 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:37,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:37,235 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,235 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:37,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:37,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:37,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 09:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38ef04ca[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:37,237 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 09:55:37,238 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:37,240 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,240 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:37,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 09:55:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 09:55:37,241 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:37,242 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:37,242 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,242 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:37,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 09:55:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 09:55:37,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:37,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:37,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:37,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 09:55:37,247 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 09:55:37,247 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:37,247 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:37,247 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 09:55:37,247 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:37,249 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,249 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,249 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,249 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:37,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,250 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:37,250 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,251 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,253 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:37,253 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 09:55:37,253 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,255 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:37,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 09:55:37,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:37,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,256 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:37,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:37,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:37,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,256 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:37,257 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,257 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 09:55:37,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,257 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 09:55:37,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:37,356 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:37,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:37,363 INFO [Listener at localhost/34597] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,365 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.StoreUtils(129): cannot split hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/5fdb0192361141c1955636e64f9a4d46 because midkey is the same as first or last row 2023-02-11 09:55:37,365 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.StoreUtils(129): cannot split hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/b67ae91f3b9e4ce4b208aaad77d0638d because midkey is the same as first or last row 2023-02-11 09:55:37,365 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.HRegion(2744): Flushing 8aaae3a9f0c9f48611203f0004ccdf43 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 09:55:37,374 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/36fd6a2cde1f4217933e4f6076ff3287 2023-02-11 09:55:37,384 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/f87006cd51734277befe911a215323ec 2023-02-11 09:55:37,387 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf1/36fd6a2cde1f4217933e4f6076ff3287 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/36fd6a2cde1f4217933e4f6076ff3287 2023-02-11 09:55:37,390 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/36fd6a2cde1f4217933e4f6076ff3287, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:37,391 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/.tmp/cf2/f87006cd51734277befe911a215323ec as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/f87006cd51734277befe911a215323ec 2023-02-11 09:55:37,394 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/f87006cd51734277befe911a215323ec, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:37,395 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8aaae3a9f0c9f48611203f0004ccdf43 in 30ms, sequenceid=82, compaction requested=true 2023-02-11 09:55:37,396 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.HRegion(2445): Flush status journal for 8aaae3a9f0c9f48611203f0004ccdf43: 2023-02-11 09:55:37,396 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.StoreUtils(129): cannot split hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/5fdb0192361141c1955636e64f9a4d46 because midkey is the same as first or last row 2023-02-11 09:55:37,396 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41949] regionserver.StoreUtils(129): cannot split hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/b67ae91f3b9e4ce4b208aaad77d0638d because midkey is the same as first or last row 2023-02-11 09:55:37,396 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,397 DEBUG [Listener at localhost/34597] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-11 09:55:37,397 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:37,399 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-11 09:55:37,399 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-11 09:55:37,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:37,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 09:55:37,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109337402"}]},"ts":"1676109337402"} 2023-02-11 09:55:37,403 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-11 09:55:37,405 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-11 09:55:37,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8aaae3a9f0c9f48611203f0004ccdf43, UNASSIGN}] 2023-02-11 09:55:37,406 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8aaae3a9f0c9f48611203f0004ccdf43, UNASSIGN 2023-02-11 09:55:37,406 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=8aaae3a9f0c9f48611203f0004ccdf43, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:37,407 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676109337406"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109337406"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109337406"}]},"ts":"1676109337406"} 2023-02-11 09:55:37,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 8aaae3a9f0c9f48611203f0004ccdf43, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:37,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 09:55:37,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:37,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8aaae3a9f0c9f48611203f0004ccdf43, disabling compactions & flushes 2023-02-11 09:55:37,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. after waiting 0 ms 2023-02-11 09:55:37,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 09:55:37,564 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:37,564 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43. 2023-02-11 09:55:37,564 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8aaae3a9f0c9f48611203f0004ccdf43: Waiting for close lock at 1676109337559Running coprocessor pre-close hooks at 1676109337559Disabling compacts and flushes for region at 1676109337559Disabling writes for close at 1676109337560 (+1 ms)Writing region close event to WAL at 1676109337561 (+1 ms)Running coprocessor post-close hooks at 1676109337564 (+3 ms)Closed at 1676109337564 2023-02-11 09:55:37,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:37,566 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=8aaae3a9f0c9f48611203f0004ccdf43, regionState=CLOSED 2023-02-11 09:55:37,566 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676109337566"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109337566"}]},"ts":"1676109337566"} 2023-02-11 09:55:37,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-11 09:55:37,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 8aaae3a9f0c9f48611203f0004ccdf43, server=jenkins-hbase4.apache.org,41949,1676109241776 in 159 msec 2023-02-11 09:55:37,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-11 09:55:37,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=8aaae3a9f0c9f48611203f0004ccdf43, UNASSIGN in 164 msec 2023-02-11 09:55:37,571 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109337571"}]},"ts":"1676109337571"} 2023-02-11 09:55:37,572 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-11 09:55:37,574 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-11 09:55:37,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 176 msec 2023-02-11 09:55:37,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 09:55:37,704 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-11 09:55:37,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-11 09:55:37,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:37,708 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:37,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 09:55:37,709 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:37,712 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:37,714 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/recovered.edits] 2023-02-11 09:55:37,718 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/1851f053ea5646d382e95217e12e6b92 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/1851f053ea5646d382e95217e12e6b92 2023-02-11 09:55:37,728 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/36fd6a2cde1f4217933e4f6076ff3287 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/36fd6a2cde1f4217933e4f6076ff3287 2023-02-11 09:55:37,730 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/453cd95e493941c89e3d44bfc46602f2 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/453cd95e493941c89e3d44bfc46602f2 2023-02-11 09:55:37,733 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/5fdb0192361141c1955636e64f9a4d46 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/5fdb0192361141c1955636e64f9a4d46 2023-02-11 09:55:37,736 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/86131bfe566f4518b50c6dc236f590da to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/86131bfe566f4518b50c6dc236f590da 2023-02-11 09:55:37,745 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/91e3f9bb019d4818970529044a6aee8d to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/91e3f9bb019d4818970529044a6aee8d 2023-02-11 09:55:37,746 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/c79b078b635a46188506ab4651bccb8a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf1/c79b078b635a46188506ab4651bccb8a 2023-02-11 09:55:37,750 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/0d7c54a28d7c41d19b853e71e9d8208b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/0d7c54a28d7c41d19b853e71e9d8208b 2023-02-11 09:55:37,752 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/5c4be7c3d810411889fdeb327952dbb9 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/5c4be7c3d810411889fdeb327952dbb9 2023-02-11 09:55:37,753 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/8509eda6e0a24bbc8ff3c4a7aeca0268 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/8509eda6e0a24bbc8ff3c4a7aeca0268 2023-02-11 09:55:37,755 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/a75ab0466cea4cfa8fc6a8ea460cab2a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/a75ab0466cea4cfa8fc6a8ea460cab2a 2023-02-11 09:55:37,756 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/b67ae91f3b9e4ce4b208aaad77d0638d to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/b67ae91f3b9e4ce4b208aaad77d0638d 2023-02-11 09:55:37,758 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/e0bbc65f74b54faea1223c9858171800 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/e0bbc65f74b54faea1223c9858171800 2023-02-11 09:55:37,759 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/f87006cd51734277befe911a215323ec to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/cf2/f87006cd51734277befe911a215323ec 2023-02-11 09:55:37,765 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/recovered.edits/85.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43/recovered.edits/85.seqid 2023-02-11 09:55:37,765 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testInvalidSplitKey/8aaae3a9f0c9f48611203f0004ccdf43 2023-02-11 09:55:37,765 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 09:55:37,768 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:37,770 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-11 09:55:37,774 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-11 09:55:37,775 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:37,776 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-11 09:55:37,776 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109337776"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:37,778 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 09:55:37,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8aaae3a9f0c9f48611203f0004ccdf43, NAME => 'testInvalidSplitKey,,1676109335764.8aaae3a9f0c9f48611203f0004ccdf43.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 09:55:37,778 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-11 09:55:37,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109337778"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:37,779 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-11 09:55:37,781 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:37,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 76 msec 2023-02-11 09:55:37,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 09:55:37,810 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-11 09:55:37,827 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 347) - Thread LEAK? -, OpenFileDescriptor=745 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=105 (was 105), ProcessCount=174 (was 173) - ProcessCount LEAK? -, AvailableMemoryMB=2526 (was 2555) 2023-02-11 09:55:37,842 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=745, MaxFileDescriptor=60000, SystemLoadAverage=105, ProcessCount=174, AvailableMemoryMB=2526 2023-02-11 09:55:37,842 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:37,845 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:55:37,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:55:37,847 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:37,849 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:55:37,849 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:55:37,851 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:37,852 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab empty. 2023-02-11 09:55:37,852 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:37,852 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 09:55:37,873 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 09:55:37,874 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2d09da8c5c6e64b92b6df9ca89193aab, NAME => 'testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:55:37,882 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:37,882 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 2d09da8c5c6e64b92b6df9ca89193aab, disabling compactions & flushes 2023-02-11 09:55:37,882 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:37,882 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:37,882 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. after waiting 0 ms 2023-02-11 09:55:37,882 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:37,882 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:37,882 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 2d09da8c5c6e64b92b6df9ca89193aab: Waiting for close lock at 1676109337882Disabling compacts and flushes for region at 1676109337882Disabling writes for close at 1676109337882Writing region close event to WAL at 1676109337882Closed at 1676109337882 2023-02-11 09:55:37,884 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:55:37,885 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109337885"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109337885"}]},"ts":"1676109337885"} 2023-02-11 09:55:37,886 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:55:37,887 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:55:37,887 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109337887"}]},"ts":"1676109337887"} 2023-02-11 09:55:37,888 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 09:55:37,892 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:55:37,893 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:55:37,893 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:55:37,893 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:55:37,893 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:55:37,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, ASSIGN}] 2023-02-11 09:55:37,895 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, ASSIGN 2023-02-11 09:55:37,895 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:55:38,046 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:55:38,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=2d09da8c5c6e64b92b6df9ca89193aab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109338047"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109338047"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109338047"}]},"ts":"1676109338047"} 2023-02-11 09:55:38,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:38,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2d09da8c5c6e64b92b6df9ca89193aab, NAME => 'testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:55:38,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:38,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:38,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,205 INFO [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,206 DEBUG [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1 2023-02-11 09:55:38,206 DEBUG [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1 2023-02-11 09:55:38,206 INFO [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d09da8c5c6e64b92b6df9ca89193aab columnFamilyName cf1 2023-02-11 09:55:38,207 INFO [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] regionserver.HStore(310): Store=2d09da8c5c6e64b92b6df9ca89193aab/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:38,207 INFO [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,208 DEBUG [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2 2023-02-11 09:55:38,208 DEBUG [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2 2023-02-11 09:55:38,208 INFO [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d09da8c5c6e64b92b6df9ca89193aab columnFamilyName cf2 2023-02-11 09:55:38,209 INFO [StoreOpener-2d09da8c5c6e64b92b6df9ca89193aab-1] regionserver.HStore(310): Store=2d09da8c5c6e64b92b6df9ca89193aab/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:38,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:38,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:38,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:55:38,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2d09da8c5c6e64b92b6df9ca89193aab; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70864031, jitterRate=0.05595634877681732}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:38,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2d09da8c5c6e64b92b6df9ca89193aab: Running coprocessor pre-open hook at 1676109338204Writing region info on filesystem at 1676109338204Initializing all the Stores at 1676109338205 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109338205Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109338205Cleaning up temporary data from old regions at 1676109338210 (+5 ms)Cleaning up detritus from prior splits at 1676109338211 (+1 ms)Running coprocessor post-open hooks at 1676109338214 (+3 ms)Region opened successfully at 1676109338214 2023-02-11 09:55:38,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab., pid=111, masterSystemTime=1676109338200 2023-02-11 09:55:38,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=2d09da8c5c6e64b92b6df9ca89193aab, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,217 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109338216"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109338216"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109338216"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109338216"}]},"ts":"1676109338216"} 2023-02-11 09:55:38,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-11 09:55:38,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 in 169 msec 2023-02-11 09:55:38,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-11 09:55:38,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, ASSIGN in 326 msec 2023-02-11 09:55:38,221 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:55:38,221 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109338221"}]},"ts":"1676109338221"} 2023-02-11 09:55:38,222 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 09:55:38,224 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:55:38,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 378 msec 2023-02-11 09:55:38,351 ERROR [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid b979c166-8ceb-4cd3-99a3-1b2b2b1e0014) service to localhost/127.0.0.1:41137 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:55:38,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:38,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 09:55:38,356 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,356 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,356 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 09:55:38,356 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:55:38,357 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,357 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,359 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,359 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,359 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,359 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,360 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,360 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:38,361 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,361 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,361 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. started... 2023-02-11 09:55:38,362 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2d09da8c5c6e64b92b6df9ca89193aab 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:55:38,363 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,363 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,364 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,364 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,364 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,364 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,371 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/d841a697a87847ba903061e8e5cb66ff 2023-02-11 09:55:38,388 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/996013109916492c8436a2bfcfb41ec7 2023-02-11 09:55:38,393 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/d841a697a87847ba903061e8e5cb66ff as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff 2023-02-11 09:55:38,397 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:38,400 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/996013109916492c8436a2bfcfb41ec7 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7 2023-02-11 09:55:38,403 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:38,404 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 42ms, sequenceid=6, compaction requested=false 2023-02-11 09:55:38,404 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,404 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2d09da8c5c6e64b92b6df9ca89193aab: 2023-02-11 09:55:38,404 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,404 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,406 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,406 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,407 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,407 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,409 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 09:55:38,409 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@226491d4[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:38,409 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:55:38,409 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,410 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:38,411 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,411 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:38,411 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:38,414 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,414 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,414 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,414 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,414 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,415 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,415 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,415 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:38,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:55:38,419 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 09:55:38,419 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:38,419 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:38,419 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,419 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:38,421 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,422 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,422 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,423 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,425 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,426 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,426 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,436 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:38,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,437 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:38,436 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,437 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,437 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,437 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,437 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,437 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,437 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,438 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,537 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:38,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:38,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:38,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:38,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 09:55:38,550 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,550 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,550 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 09:55:38,551 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:55:38,551 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,551 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,552 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,552 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,552 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,552 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,553 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,553 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,554 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,554 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,554 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,554 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,554 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,554 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,554 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,554 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,554 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. started... 2023-02-11 09:55:38,555 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2d09da8c5c6e64b92b6df9ca89193aab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:38,557 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,558 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,558 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,558 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,558 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,558 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,565 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/5ae009ebf178474d93dd666d49a47968 2023-02-11 09:55:38,575 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 2023-02-11 09:55:38,578 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/5ae009ebf178474d93dd666d49a47968 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968 2023-02-11 09:55:38,581 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:38,581 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 2023-02-11 09:55:38,584 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:38,585 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 30ms, sequenceid=19, compaction requested=false 2023-02-11 09:55:38,586 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2d09da8c5c6e64b92b6df9ca89193aab: 2023-02-11 09:55:38,586 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,586 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:38,586 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:38,586 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,586 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,586 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,590 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,590 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,590 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,590 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 09:55:38,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47ed2ed1[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:38,592 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:55:38,592 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,599 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:38,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,599 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:38,600 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,600 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,600 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:38,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:38,601 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,601 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,601 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,601 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,602 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,602 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,602 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,602 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,602 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,602 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:38,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:55:38,606 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 09:55:38,606 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:38,606 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:38,606 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,606 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:38,607 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,607 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,607 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,607 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,607 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,607 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,608 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,608 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,608 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,609 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,611 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,611 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,612 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:38,619 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,619 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:38,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,620 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,621 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,720 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:38,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:38,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:38,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:38,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 09:55:38,733 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 09:55:38,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:55:38,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,734 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,735 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,734 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,734 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,735 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,735 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,737 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,737 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,737 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:38,737 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:38,737 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. started... 2023-02-11 09:55:38,737 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,737 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,737 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,737 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2d09da8c5c6e64b92b6df9ca89193aab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:38,738 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,739 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,739 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,739 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,739 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,739 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,745 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 2023-02-11 09:55:38,754 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 2023-02-11 09:55:38,758 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 2023-02-11 09:55:38,762 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:38,763 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 2023-02-11 09:55:38,766 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:38,767 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 30ms, sequenceid=32, compaction requested=true 2023-02-11 09:55:38,767 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2d09da8c5c6e64b92b6df9ca89193aab: 2023-02-11 09:55:38,767 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,770 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,770 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,771 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,771 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 09:55:38,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c776683[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:38,773 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:55:38,773 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,774 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:38,776 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,776 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,776 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:38,776 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,776 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,776 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,776 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,776 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,777 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,776 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,777 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,777 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:38,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:55:38,779 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 09:55:38,779 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:38,779 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:38,779 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,779 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:38,782 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,782 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,782 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,782 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,782 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,783 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,784 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,785 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,785 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,789 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,789 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,789 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,789 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,790 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,789 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:38,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:38,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,791 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,790 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:38,791 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,791 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,791 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,891 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:38,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:38,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:38,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:38,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 09:55:38,903 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,903 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,903 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 09:55:38,903 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:55:38,904 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,904 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,905 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,905 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,905 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,905 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,905 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,906 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:38,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:38,906 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:38,907 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:38,907 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,907 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. started... 2023-02-11 09:55:38,908 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2d09da8c5c6e64b92b6df9ca89193aab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:38,910 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,910 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,910 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,910 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,911 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,911 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,916 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd 2023-02-11 09:55:38,927 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/ec1fd141bd7c4a898ef345f134fd26b3 2023-02-11 09:55:38,930 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd 2023-02-11 09:55:38,933 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:38,934 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/ec1fd141bd7c4a898ef345f134fd26b3 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3 2023-02-11 09:55:38,936 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:38,937 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 29ms, sequenceid=45, compaction requested=true 2023-02-11 09:55:38,937 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2d09da8c5c6e64b92b6df9ca89193aab: 2023-02-11 09:55:38,937 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:38,937 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:38,937 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:38,937 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,937 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:38,937 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:38,939 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,939 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,939 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,939 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:38,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 09:55:38,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c7ee293[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:38,941 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:55:38,941 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,942 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,942 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,942 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:38,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,942 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:38,943 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,943 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,943 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:38,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,943 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:38,943 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:38,943 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:38,945 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,945 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,945 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,945 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,946 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:38,946 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,946 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,946 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:38,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:55:38,948 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 09:55:38,948 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:38,948 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:38,949 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 09:55:38,949 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:38,950 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,950 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:38,951 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:38,951 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:38,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,952 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,953 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:38,953 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:38,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:38,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:38,955 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:38,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:38,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:38,956 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:38,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,956 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:38,957 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:38,957 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,957 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:38,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:38,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,056 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:39,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:39,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:39,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:39,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 09:55:39,069 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:39,069 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,069 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 09:55:39,069 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:55:39,069 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,070 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,071 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,071 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,071 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:39,071 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:39,072 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,072 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:39,072 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:39,072 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:39,072 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:39,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:39,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:39,073 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:39,073 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:39,073 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. started... 2023-02-11 09:55:39,073 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2d09da8c5c6e64b92b6df9ca89193aab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:39,074 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,074 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,074 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:39,077 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,077 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,077 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:39,083 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/6d80e8b5ef06418793318cbbd4f9a227 2023-02-11 09:55:39,093 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 2023-02-11 09:55:39,096 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/6d80e8b5ef06418793318cbbd4f9a227 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227 2023-02-11 09:55:39,099 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:39,100 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 2023-02-11 09:55:39,103 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:39,103 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 30ms, sequenceid=58, compaction requested=true 2023-02-11 09:55:39,103 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2d09da8c5c6e64b92b6df9ca89193aab: 2023-02-11 09:55:39,103 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:39,103 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:39,103 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:39,103 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,103 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:39,104 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:39,108 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,108 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:39,109 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,109 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 09:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a1238f9[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:39,111 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:55:39,111 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:39,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:39,112 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,112 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:39,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:39,112 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:39,115 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,115 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:39,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:39,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:39,115 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,115 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:39,116 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:39,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:39,116 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,116 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:39,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:39,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:39,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:39,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:55:39,118 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 09:55:39,118 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:39,118 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:39,118 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,118 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:39,120 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:39,120 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:39,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,120 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,121 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,121 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:39,122 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,123 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,123 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,127 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:39,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:39,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,127 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,128 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,128 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:39,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,128 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,128 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,128 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,128 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,228 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:39,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:39,239 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:39,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:39,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 09:55:39,241 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:39,241 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,241 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 09:55:39,241 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 09:55:39,241 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,241 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,242 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,242 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,242 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,242 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,243 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,243 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:39,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:39,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:39,244 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 09:55:39,244 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:39,244 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:39,244 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:39,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,244 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:39,245 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:39,245 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:39,245 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. started... 2023-02-11 09:55:39,245 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2d09da8c5c6e64b92b6df9ca89193aab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:39,246 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,246 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,246 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,247 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:39,247 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,247 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:39,253 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/7dd81a7911244521bf2ca03495cc4264 2023-02-11 09:55:39,263 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/22f0483465d6436db54645868ab3d499 2023-02-11 09:55:39,267 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/7dd81a7911244521bf2ca03495cc4264 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264 2023-02-11 09:55:39,270 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:39,270 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/22f0483465d6436db54645868ab3d499 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499 2023-02-11 09:55:39,273 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:39,274 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 29ms, sequenceid=71, compaction requested=true 2023-02-11 09:55:39,274 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2d09da8c5c6e64b92b6df9ca89193aab: 2023-02-11 09:55:39,274 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:39,274 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:39,274 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:39,274 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,274 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 09:55:39,274 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 09:55:39,276 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,276 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:39,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:39,277 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,277 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:39,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:39,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:39,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:39,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 09:55:39,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24a714cc[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:39,278 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 09:55:39,278 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,279 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:39,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:39,280 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,280 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:39,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 09:55:39,280 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:39,281 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:39,281 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,281 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:39,282 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,282 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:39,282 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:39,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:39,282 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:39,284 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 09:55:39,284 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,284 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 09:55:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:39,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 09:55:39,286 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 09:55:39,286 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:39,286 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:39,286 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 09:55:39,286 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:39,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:39,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,288 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,288 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:39,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,289 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,289 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:39,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:39,291 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 09:55:39,291 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,291 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 09:55:39,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,296 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:39,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:39,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:39,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:39,297 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:39,298 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,298 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 09:55:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:39,397 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:39,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:39,655 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 09:55:39,656 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:39,657 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,658 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,658 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,658 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:39,659 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:39,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:39,907 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,908 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:39,908 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:39,908 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:39,908 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-11 09:55:39,908 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:39,908 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:39,908 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:39,908 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:39,909 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:39,910 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 09:55:39,910 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:39,910 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:39,911 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:39,911 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:39,911 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:39,911 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:39,911 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:39,911 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:39,912 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:39,912 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:39,912 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:39,912 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:39,912 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:39,912 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:39,913 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:39,914 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:39,914 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:39,914 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:39,914 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:39,914 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:39,915 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:39,915 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:39,915 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:39,915 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:39,915 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:39,915 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:39,915 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:39,916 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,916 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:39,916 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,916 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,916 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,916 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 09:55:39,916 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:39,917 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:39,917 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109369917 2023-02-11 09:55:39,919 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,919 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:39,921 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:39,921 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:39,921 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:39,921 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109399921 2023-02-11 09:55:39,921 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109459921 2023-02-11 09:55:39,921 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:55:39,922 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:39,923 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:39,923 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:39,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:39,923 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:40,172 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:40,172 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:40,173 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:40,174 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 09:55:40,175 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:40,175 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:40,175 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:40,175 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:40,176 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,176 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:40,176 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,176 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,176 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,176 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:40,177 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,177 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,177 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,177 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:40,177 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:40,178 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,178 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,178 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,178 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:40,178 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,178 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,178 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,179 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,180 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,180 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,180 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,180 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:40,181 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,181 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:40,181 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,181 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,181 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,181 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-11 09:55:40,182 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:40,182 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:40,183 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109370183 2023-02-11 09:55:40,184 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,184 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:40,186 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:40,186 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:40,186 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:40,186 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109400186 2023-02-11 09:55:40,186 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109460186 2023-02-11 09:55:40,187 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:55:40,187 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:40,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN}] 2023-02-11 09:55:40,189 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:40,189 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:40,189 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:40,437 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:40,437 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:40,438 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:40,439 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 09:55:40,439 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:40,440 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:40,440 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:40,440 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:40,440 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,440 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:40,440 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,441 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,441 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,441 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:40,441 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,441 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,441 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,441 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,442 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,443 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,444 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,444 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,444 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,444 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:40,444 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,444 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:40,444 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,445 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,445 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,445 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:55:40,445 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:40,446 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:40,446 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109370446 2023-02-11 09:55:40,447 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:40,447 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,450 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:40,450 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:40,450 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:40,450 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109400450 2023-02-11 09:55:40,450 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109460450 2023-02-11 09:55:40,450 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:55:40,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN}] 2023-02-11 09:55:40,450 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:40,452 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:40,453 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:40,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109340453"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109340453"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109340453"}]},"ts":"1676109340453"} 2023-02-11 09:55:40,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:40,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2d09da8c5c6e64b92b6df9ca89193aab, disabling compactions & flushes 2023-02-11 09:55:40,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:40,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:40,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. after waiting 0 ms 2023-02-11 09:55:40,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:40,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2d09da8c5c6e64b92b6df9ca89193aab 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 09:55:40,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/0090de63230e480488b49a157f4d5e90 2023-02-11 09:55:40,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/c97c25476032483e99937b7bd569ae05 2023-02-11 09:55:40,627 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf1/0090de63230e480488b49a157f4d5e90 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90 2023-02-11 09:55:40,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:40,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/.tmp/cf2/c97c25476032483e99937b7bd569ae05 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05 2023-02-11 09:55:40,634 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:40,635 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 29ms, sequenceid=82, compaction requested=true 2023-02-11 09:55:40,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 09:55:40,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:40,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. 2023-02-11 09:55:40,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2d09da8c5c6e64b92b6df9ca89193aab: Waiting for close lock at 1676109340606Running coprocessor pre-close hooks at 1676109340606Disabling compacts and flushes for region at 1676109340606Disabling writes for close at 1676109340606Obtaining lock to block concurrent updates at 1676109340606Preparing flush snapshotting stores in 2d09da8c5c6e64b92b6df9ca89193aab at 1676109340606Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676109340606Flushing stores of testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. at 1676109340607 (+1 ms)Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf1: creating writer at 1676109340607Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf1: appending metadata at 1676109340608 (+1 ms)Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf1: closing flushed file at 1676109340608Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf2: creating writer at 1676109340617 (+9 ms)Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf2: appending metadata at 1676109340618 (+1 ms)Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf2: closing flushed file at 1676109340618Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf1: reopening flushed file at 1676109340627 (+9 ms)Flushing 2d09da8c5c6e64b92b6df9ca89193aab/cf2: reopening flushed file at 1676109340631 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2d09da8c5c6e64b92b6df9ca89193aab in 29ms, sequenceid=82, compaction requested=true at 1676109340635 (+4 ms)Writing region close event to WAL at 1676109340636 (+1 ms)Running coprocessor post-close hooks at 1676109340639 (+3 ms)Closed at 1676109340640 (+1 ms) 2023-02-11 09:55:40,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,642 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED 2023-02-11 09:55:40,642 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109340642"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109340642"}]},"ts":"1676109340642"} 2023-02-11 09:55:40,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-11 09:55:40,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 in 189 msec 2023-02-11 09:55:40,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-11 09:55:40,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN in 194 msec 2023-02-11 09:55:40,647 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:40,648 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:40,648 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:40,700 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,700 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:40,700 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:40,700 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:40,701 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:40,701 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:40,701 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:40,701 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:40,701 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:40,701 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:40,702 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 09:55:40,702 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:40,702 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:40,703 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:40,703 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:40,703 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:40,703 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:40,703 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,703 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:40,704 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,704 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:40,704 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,704 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:40,704 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,704 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:40,704 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,705 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:40,706 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,706 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:40,706 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,706 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:40,706 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,706 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,707 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:40,707 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,707 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:40,707 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,707 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,707 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:40,707 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:40,708 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,708 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:40,708 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:40,708 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:40,708 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,708 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,708 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,708 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:40,708 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:40,708 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:55:40,709 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:40,709 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:40,709 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109370709 2023-02-11 09:55:40,710 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:40,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:40,714 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:40,714 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:40,714 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:40,714 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109400714 2023-02-11 09:55:40,714 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109460714 2023-02-11 09:55:40,714 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:55:40,716 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:40,723 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=2d09da8c5c6e64b92b6df9ca89193aab, threads=14 2023-02-11 09:55:40,723 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,723 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,724 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,724 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,724 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,724 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,724 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,724 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,724 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,725 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,725 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,725 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,725 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,726 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,901 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,902 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,914 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,915 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,928 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,931 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,936 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,938 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,943 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,946 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,946 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:40,946 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,035 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:55:41,296 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,296 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,296 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 2d09da8c5c6e64b92b6df9ca89193aab Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:55:41,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:41,321 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:41,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:41,464 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,464 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:41,465 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:41,465 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:41,465 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:41,465 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:41,465 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:41,465 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:41,465 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:41,466 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:41,467 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 09:55:41,467 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:41,467 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:41,467 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:41,468 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:41,468 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:41,468 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:41,468 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,468 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:41,468 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,469 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,469 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,469 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:41,469 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,469 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,469 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,469 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,470 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,471 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:41,472 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:41,472 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,473 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,473 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,473 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:41,473 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:41,473 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:55:41,473 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:41,474 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:41,474 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109371474 2023-02-11 09:55:41,476 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,476 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:41,479 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:41,479 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:41,479 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:41,479 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109401479 2023-02-11 09:55:41,479 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109461479 2023-02-11 09:55:41,479 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:55:41,488 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=2d09da8c5c6e64b92b6df9ca89193aab, threads=14 2023-02-11 09:55:41,488 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,488 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,490 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,490 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,492 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,492 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,492 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,493 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,493 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,496 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,496 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,496 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,497 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,497 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,585 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,586 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,592 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,592 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,594 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,596 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,597 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,597 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,600 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,600 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,600 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,602 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,602 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,603 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264 for region: 2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:41,603 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 2d09da8c5c6e64b92b6df9ca89193aab Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:55:41,630 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:41,634 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:55:41,635 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:55:41,635 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:41,635 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:41,636 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:41,730 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:41,730 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:41,731 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:41,732 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 09:55:41,732 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:41,732 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:41,732 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:41,733 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:41,733 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:41,733 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:41,733 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,733 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:41,733 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,733 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:41,734 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:41,735 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,736 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,736 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,736 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,736 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:41,736 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,736 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,736 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:41,737 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:41,737 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:41,737 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:41,737 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,737 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:41,737 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:41,737 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:41,738 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,738 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,738 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,738 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:41,738 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:41,738 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:55:41,738 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:41,739 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:41,739 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109371739 2023-02-11 09:55:41,739 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:41,741 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,743 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:41,743 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:41,743 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:41,743 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109401743 2023-02-11 09:55:41,743 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109461743 2023-02-11 09:55:41,743 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:55:41,746 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:41,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:41,747 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:41,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:41,993 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:41,994 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:41,994 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:41,994 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:41,994 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:41,994 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:41,994 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:41,994 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:41,994 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:41,995 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:41,996 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 09:55:41,996 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:41,996 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:41,996 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:41,996 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:41,996 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:41,997 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:41,998 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,999 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:41,999 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,999 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:41,999 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,999 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:41,999 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:41,999 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,000 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:42,000 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,000 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,000 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,000 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,000 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,000 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:42,001 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,001 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:42,001 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:42,001 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:42,001 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,001 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,001 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,001 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,001 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,001 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:55:42,002 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:42,002 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:42,003 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109372003 2023-02-11 09:55:42,003 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:42,004 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,006 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:42,006 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:42,007 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:42,007 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109402007 2023-02-11 09:55:42,007 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109462007 2023-02-11 09:55:42,007 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 09:55:42,007 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:42,009 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109342009"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676109342009"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676109342009"}]},"ts":"1676109342009"} 2023-02-11 09:55:42,009 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109342009"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109342009"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109342009"}]},"ts":"1676109342009"} 2023-02-11 09:55:42,009 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109342009"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109342009"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109342009"}]},"ts":"1676109342009"} 2023-02-11 09:55:42,010 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:42,010 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:42,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:42,257 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,257 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:42,257 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:42,257 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:42,257 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:42,257 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:42,257 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:42,257 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:42,258 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:42,258 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:42,259 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 09:55:42,259 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:42,259 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:42,259 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:42,260 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:42,260 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:42,260 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:42,260 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,260 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:42,260 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,261 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,261 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,261 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:42,261 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,261 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,261 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,262 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:42,262 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:42,262 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,262 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,262 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,262 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:42,262 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,263 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,263 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,263 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:42,263 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,263 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,263 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,264 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,264 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:42,264 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,264 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,264 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,264 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,264 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,265 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:42,265 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,265 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:42,265 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:42,265 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:42,265 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,265 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,265 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,265 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,265 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,266 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 09:55:42,266 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:42,267 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:42,267 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109372267 2023-02-11 09:55:42,269 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:42,269 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,271 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:42,271 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:42,271 INFO [Listener at localhost/34597] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:42,271 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 09:55:42,271 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=304850c3fd2c34006e777e4aba5f33e0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 09:55:42,271 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:42,271 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109402271 2023-02-11 09:55:42,272 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109462272 2023-02-11 09:55:42,272 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 09:55:42,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676109342272"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676109342272"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676109342272"}]},"ts":"1676109342272"} 2023-02-11 09:55:42,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109342272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109342272"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109342272"}]},"ts":"1676109342272"} 2023-02-11 09:55:42,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109342272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109342272"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109342272"}]},"ts":"1676109342272"} 2023-02-11 09:55:42,274 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:42,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:42,275 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:42,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:42,522 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:42,522 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:42,523 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:42,524 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-11 09:55:42,524 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:42,524 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:42,524 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:42,525 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:42,525 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:42,525 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:42,525 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,525 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:42,525 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:42,526 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:42,527 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,528 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,528 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,528 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,528 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:42,528 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,528 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,528 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,529 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,529 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,529 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:42,529 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,529 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:42,529 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:42,529 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:42,529 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,529 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,529 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,530 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,530 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,530 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:55:42,530 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:42,531 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:42,531 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109372531 2023-02-11 09:55:42,531 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:42,533 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,535 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:42,535 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:42,536 INFO [Listener at localhost/34597] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:42,536 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 09:55:42,536 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=304850c3fd2c34006e777e4aba5f33e0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 09:55:42,536 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:42,536 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109402536 2023-02-11 09:55:42,536 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109462536 2023-02-11 09:55:42,536 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 09:55:42,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:42,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, ASSIGN}] 2023-02-11 09:55:42,539 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:42,539 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:42,539 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:42,786 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,787 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:42,787 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:42,787 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:42,787 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:42,787 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:42,787 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:42,787 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:42,787 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:42,788 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:42,789 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-11 09:55:42,789 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:42,789 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:42,789 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:42,790 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:42,790 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:42,790 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:42,790 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,791 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:42,791 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,791 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:42,791 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,791 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:42,791 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,791 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:42,792 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,793 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:42,793 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,793 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:42,793 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,793 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:42,793 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,793 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:42,794 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,795 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:42,795 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:42,795 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:42,795 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,795 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,795 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,795 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,795 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:42,795 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 09:55:42,796 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:42,796 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:42,796 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109372796 2023-02-11 09:55:42,796 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:42,798 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:42,800 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:42,800 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:42,800 INFO [Listener at localhost/34597] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:42,801 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 09:55:42,801 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=304850c3fd2c34006e777e4aba5f33e0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 09:55:42,801 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:42,801 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109402801 2023-02-11 09:55:42,801 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109462801 2023-02-11 09:55:42,801 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 09:55:42,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, ASSIGN}] 2023-02-11 09:55:42,802 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:42,803 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, ASSIGN 2023-02-11 09:55:42,804 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:55:42,804 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, ASSIGN 2023-02-11 09:55:42,805 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:55:42,954 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 09:55:42,956 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:42,956 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109342956"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109342956"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109342956"}]},"ts":"1676109342956"} 2023-02-11 09:55:42,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure a8470dda2cae9ecdee54ef2ef9721bcc, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:42,959 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=304850c3fd2c34006e777e4aba5f33e0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:42,960 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109342959"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109342959"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109342959"}]},"ts":"1676109342959"} 2023-02-11 09:55:42,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 304850c3fd2c34006e777e4aba5f33e0, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:43,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a8470dda2cae9ecdee54ef2ef9721bcc, NAME => 'testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 09:55:43,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:43,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:43,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,113 INFO [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,114 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1 2023-02-11 09:55:43,114 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1 2023-02-11 09:55:43,114 INFO [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8470dda2cae9ecdee54ef2ef9721bcc columnFamilyName cf1 2023-02-11 09:55:43,126 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90-bottom 2023-02-11 09:55:43,129 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd-bottom 2023-02-11 09:55:43,132 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968-bottom 2023-02-11 09:55:43,136 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227-bottom 2023-02-11 09:55:43,138 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264-bottom 2023-02-11 09:55:43,141 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff-bottom 2023-02-11 09:55:43,144 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147-bottom 2023-02-11 09:55:43,144 INFO [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(310): Store=a8470dda2cae9ecdee54ef2ef9721bcc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:43,144 INFO [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,145 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2 2023-02-11 09:55:43,145 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2 2023-02-11 09:55:43,145 INFO [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8470dda2cae9ecdee54ef2ef9721bcc columnFamilyName cf2 2023-02-11 09:55:43,156 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499-bottom 2023-02-11 09:55:43,159 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759-bottom 2023-02-11 09:55:43,162 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0-bottom 2023-02-11 09:55:43,166 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54-bottom 2023-02-11 09:55:43,169 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7-bottom 2023-02-11 09:55:43,172 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05-bottom 2023-02-11 09:55:43,175 DEBUG [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3-bottom 2023-02-11 09:55:43,175 INFO [StoreOpener-a8470dda2cae9ecdee54ef2ef9721bcc-1] regionserver.HStore(310): Store=a8470dda2cae9ecdee54ef2ef9721bcc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:43,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:43,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a8470dda2cae9ecdee54ef2ef9721bcc; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69115183, jitterRate=0.029896482825279236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:43,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a8470dda2cae9ecdee54ef2ef9721bcc: Running coprocessor pre-open hook at 1676109343112Writing region info on filesystem at 1676109343112Initializing all the Stores at 1676109343113 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109343113Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109343113Cleaning up temporary data from old regions at 1676109343178 (+65 ms)Cleaning up detritus from prior splits at 1676109343178Running coprocessor post-open hooks at 1676109343181 (+3 ms)Region opened successfully at 1676109343181 2023-02-11 09:55:43,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc., pid=117, masterSystemTime=1676109343109 2023-02-11 09:55:43,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:43,183 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:43,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:55:43,183 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:43,183 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,183 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1912): a8470dda2cae9ecdee54ef2ef9721bcc/cf1 is initiating minor compaction (all files) 2023-02-11 09:55:43,183 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,183 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1912): a8470dda2cae9ecdee54ef2ef9721bcc/cf2 is initiating minor compaction (all files) 2023-02-11 09:55:43,183 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a8470dda2cae9ecdee54ef2ef9721bcc/cf1 in testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,183 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a8470dda2cae9ecdee54ef2ef9721bcc/cf2 in testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,183 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/.tmp, totalSize=34.9 K 2023-02-11 09:55:43,184 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/.tmp, totalSize=34.9 K 2023-02-11 09:55:43,184 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109338350 2023-02-11 09:55:43,184 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109338350 2023-02-11 09:55:43,184 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109338539 2023-02-11 09:55:43,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,185 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109338539 2023-02-11 09:55:43,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 304850c3fd2c34006e777e4aba5f33e0, NAME => 'testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 09:55:43,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:43,185 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109338722 2023-02-11 09:55:43,185 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109338722 2023-02-11 09:55:43,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109343185"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109343185"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109343185"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109343185"}]},"ts":"1676109343185"} 2023-02-11 09:55:43,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:43,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,186 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109338892 2023-02-11 09:55:43,186 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109338892 2023-02-11 09:55:43,187 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109339058 2023-02-11 09:55:43,187 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109339058 2023-02-11 09:55:43,187 INFO [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,187 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109339229 2023-02-11 09:55:43,188 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109339229 2023-02-11 09:55:43,188 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109339399 2023-02-11 09:55:43,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-11 09:55:43,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure a8470dda2cae9ecdee54ef2ef9721bcc, server=jenkins-hbase4.apache.org,40095,1676109241701 in 230 msec 2023-02-11 09:55:43,189 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109339399 2023-02-11 09:55:43,191 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1 2023-02-11 09:55:43,191 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1 2023-02-11 09:55:43,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, ASSIGN in 387 msec 2023-02-11 09:55:43,191 INFO [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 304850c3fd2c34006e777e4aba5f33e0 columnFamilyName cf1 2023-02-11 09:55:43,203 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] throttle.PressureAwareThroughputController(145): a8470dda2cae9ecdee54ef2ef9721bcc#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:43,203 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] throttle.PressureAwareThroughputController(145): a8470dda2cae9ecdee54ef2ef9721bcc#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:43,205 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90-top 2023-02-11 09:55:43,210 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd-top 2023-02-11 09:55:43,217 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968-top 2023-02-11 09:55:43,222 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227-top 2023-02-11 09:55:43,226 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/.tmp/cf1/4c3f88fb8f644223a6d389b863c96aa4 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/4c3f88fb8f644223a6d389b863c96aa4 2023-02-11 09:55:43,226 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264-top 2023-02-11 09:55:43,227 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/.tmp/cf2/2df90551701448e0ac9818abe114ebdd as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/2df90551701448e0ac9818abe114ebdd 2023-02-11 09:55:43,229 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff-top 2023-02-11 09:55:43,231 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a8470dda2cae9ecdee54ef2ef9721bcc/cf2 of a8470dda2cae9ecdee54ef2ef9721bcc into 2df90551701448e0ac9818abe114ebdd(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:43,232 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a8470dda2cae9ecdee54ef2ef9721bcc/cf1 of a8470dda2cae9ecdee54ef2ef9721bcc into 4c3f88fb8f644223a6d389b863c96aa4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:43,232 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a8470dda2cae9ecdee54ef2ef9721bcc: 2023-02-11 09:55:43,232 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a8470dda2cae9ecdee54ef2ef9721bcc: 2023-02-11 09:55:43,232 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc., storeName=a8470dda2cae9ecdee54ef2ef9721bcc/cf2, priority=9, startTime=1676109343183; duration=0sec 2023-02-11 09:55:43,232 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc., storeName=a8470dda2cae9ecdee54ef2ef9721bcc/cf1, priority=9, startTime=1676109343182; duration=0sec 2023-02-11 09:55:43,232 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:43,232 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:43,233 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147-top 2023-02-11 09:55:43,234 INFO [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(310): Store=304850c3fd2c34006e777e4aba5f33e0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:43,234 INFO [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,234 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2 2023-02-11 09:55:43,235 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2 2023-02-11 09:55:43,235 INFO [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 304850c3fd2c34006e777e4aba5f33e0 columnFamilyName cf2 2023-02-11 09:55:43,249 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499-top 2023-02-11 09:55:43,252 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759-top 2023-02-11 09:55:43,255 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0-top 2023-02-11 09:55:43,258 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54-top 2023-02-11 09:55:43,261 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7-top 2023-02-11 09:55:43,264 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05-top 2023-02-11 09:55:43,266 DEBUG [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3-top 2023-02-11 09:55:43,266 INFO [StoreOpener-304850c3fd2c34006e777e4aba5f33e0-1] regionserver.HStore(310): Store=304850c3fd2c34006e777e4aba5f33e0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:43,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:43,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 304850c3fd2c34006e777e4aba5f33e0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63716624, jitterRate=-0.05054831504821777}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:43,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 304850c3fd2c34006e777e4aba5f33e0: Running coprocessor pre-open hook at 1676109343186Writing region info on filesystem at 1676109343186Initializing all the Stores at 1676109343187 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109343187Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109343187Cleaning up temporary data from old regions at 1676109343268 (+81 ms)Cleaning up detritus from prior splits at 1676109343268Running coprocessor post-open hooks at 1676109343270 (+2 ms)Region opened successfully at 1676109343270 2023-02-11 09:55:43,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0., pid=118, masterSystemTime=1676109343109 2023-02-11 09:55:43,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:43,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:55:43,272 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:43,272 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:43,272 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,272 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,272 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1912): 304850c3fd2c34006e777e4aba5f33e0/cf1 is initiating minor compaction (all files) 2023-02-11 09:55:43,272 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1912): 304850c3fd2c34006e777e4aba5f33e0/cf2 is initiating minor compaction (all files) 2023-02-11 09:55:43,272 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 304850c3fd2c34006e777e4aba5f33e0/cf1 in testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,272 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 304850c3fd2c34006e777e4aba5f33e0/cf2 in testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,273 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90-top] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/.tmp, totalSize=34.9 K 2023-02-11 09:55:43,273 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05-top] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/.tmp, totalSize=34.9 K 2023-02-11 09:55:43,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,273 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109338350 2023-02-11 09:55:43,273 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109338350 2023-02-11 09:55:43,274 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=304850c3fd2c34006e777e4aba5f33e0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,274 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109343274"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109343274"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109343274"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109343274"}]},"ts":"1676109343274"} 2023-02-11 09:55:43,274 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109338539 2023-02-11 09:55:43,274 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109338539 2023-02-11 09:55:43,274 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109338722 2023-02-11 09:55:43,274 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109338722 2023-02-11 09:55:43,275 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109338892 2023-02-11 09:55:43,275 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109338892 2023-02-11 09:55:43,275 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676109339058 2023-02-11 09:55:43,275 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676109339058 2023-02-11 09:55:43,276 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting 22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676109339229 2023-02-11 09:55:43,276 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676109339229 2023-02-11 09:55:43,276 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109339399 2023-02-11 09:55:43,276 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting 0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109339399 2023-02-11 09:55:43,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-11 09:55:43,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 304850c3fd2c34006e777e4aba5f33e0, server=jenkins-hbase4.apache.org,40095,1676109241701 in 314 msec 2023-02-11 09:55:43,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-11 09:55:43,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, ASSIGN in 476 msec 2023-02-11 09:55:43,279 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 09:55:43,279 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:43,279 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 09:55:43,286 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] throttle.PressureAwareThroughputController(145): 304850c3fd2c34006e777e4aba5f33e0#cf2#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:43,286 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] throttle.PressureAwareThroughputController(145): 304850c3fd2c34006e777e4aba5f33e0#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:43,302 INFO [Listener at localhost/34597] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,302 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 09:55:43,302 INFO [Listener at localhost/34597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:43,302 INFO [Listener at localhost/34597] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:43,302 ERROR [Listener at localhost/34597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 09:55:43,302 INFO [Listener at localhost/34597] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 09:55:43,302 INFO [Listener at localhost/34597] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 09:55:43,302 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 09:55:43,303 INFO [Listener at localhost/34597] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 09:55:43,303 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 09:55:43,305 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-11 09:55:43,305 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 09:55:43,305 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 09:55:43,305 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-02-11 09:55:43,305 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-11 09:55:43,305 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-02-11 09:55:43,305 DEBUG [Listener at localhost/34597] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-11 09:55:43,306 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 09:55:43,306 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 09:55:43,306 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 09:55:43,306 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 09:55:43,306 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d216b600c0cb9961b975974a1b3ad218, daughterA=b2c7ebbb9b8d1a8598f01c1b9431d43c, daughterB=98077030a27e351168173ed0c22ff5a4 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4b63fe694a11cba94c671f4e6eff38f8, daughterA=a7bf2dbceab20d06148a05aabdce5a38, daughterB=846a3db876efcaea894eafcf63ddffd7 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 09:55:43,307 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6b41352f778cf0f0366b57384c1aaf98, daughterA=0aacff6a4012145ca0279099292e93be, daughterB=87f85251e995996fd01900699cf6571c 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6b41352f778cf0f0366b57384c1aaf98, ASSIGN 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b60cbf4c5accba4b5299abb770eaa0f7, daughterA=199774c1460010773c61876a45accec6, daughterB=7d4c56edfb8f163faef5374f2d1354c4 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 09:55:43,308 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:43,309 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a6cffd29aa384a14f910e924d07d1ce8, daughterA=79bd21cbba42e27ce102925ee9dc168a, daughterB=e42933689e13bfee2ea375c3661dba49 2023-02-11 09:55:43,309 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:43,309 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 09:55:43,309 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:43,309 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:43,309 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRollbackForSplitTableRegionWithReplica, region=6cedf5c3c6e687883cbad285602a2b77 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=1138c5b51866ce5ac5da9db13a373fc5, daughterA=9d92080b9335bfda8856ce9a59933959, daughterB=b54409f1317abce3dbe532b5d1b94543 2023-02-11 09:55:43,310 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:43,310 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 09:55:43,310 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:43,310 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:43,310 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 09:55:43,311 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:43,311 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/.tmp/cf2/dafef7510e6240eea62008db7adb94a9 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/dafef7510e6240eea62008db7adb94a9 2023-02-11 09:55:43,312 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,312 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/.tmp/cf1/71ecc552c20e45efad8be730f256d4bc as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/71ecc552c20e45efad8be730f256d4bc 2023-02-11 09:55:43,312 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN 2023-02-11 09:55:43,312 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,312 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, ASSIGN 2023-02-11 09:55:43,312 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, ASSIGN 2023-02-11 09:55:43,312 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure a8470dda2cae9ecdee54ef2ef9721bcc, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,313 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 304850c3fd2c34006e777e4aba5f33e0, server=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,313 DEBUG [Listener at localhost/34597] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 held the lock before restarting, call acquireLock to restore it. 2023-02-11 09:55:43,313 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,313 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,313 INFO [Listener at localhost/34597] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,313 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:43,313 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2d09da8c5c6e64b92b6df9ca89193aab, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:43,313 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:43,313 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:43,314 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure a8470dda2cae9ecdee54ef2ef9721bcc, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:43,314 DEBUG [Listener at localhost/34597] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 304850c3fd2c34006e777e4aba5f33e0, server=jenkins-hbase4.apache.org,40095,1676109241701 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 09:55:43,314 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-11 09:55:43,314 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 09:55:43,315 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34799,1676109241745, table=hbase:meta, region=1588230740 2023-02-11 09:55:43,328 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676109373328 2023-02-11 09:55:43,330 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,330 DEBUG [Listener at localhost/34597] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 09:55:43,332 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 304850c3fd2c34006e777e4aba5f33e0/cf1 of 304850c3fd2c34006e777e4aba5f33e0 into 71ecc552c20e45efad8be730f256d4bc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:43,332 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 304850c3fd2c34006e777e4aba5f33e0/cf2 of 304850c3fd2c34006e777e4aba5f33e0 into dafef7510e6240eea62008db7adb94a9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:43,332 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 304850c3fd2c34006e777e4aba5f33e0: 2023-02-11 09:55:43,332 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0., storeName=304850c3fd2c34006e777e4aba5f33e0/cf1, priority=9, startTime=1676109343271; duration=0sec 2023-02-11 09:55:43,332 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 304850c3fd2c34006e777e4aba5f33e0: 2023-02-11 09:55:43,332 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0., storeName=304850c3fd2c34006e777e4aba5f33e0/cf2, priority=9, startTime=1676109343272; duration=0sec 2023-02-11 09:55:43,332 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:43,332 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:43,334 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=8df39b0c7ffdf9ffd52d48b4d3daaa06, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34799,1676109241745, regionLocation=jenkins-hbase4.apache.org,34799,1676109241745, openSeqNum=2 2023-02-11 09:55:43,334 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=2d09da8c5c6e64b92b6df9ca89193aab, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=2 2023-02-11 09:55:43,334 INFO [Listener at localhost/34597] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40095,1676109241701, table=testRecoveryAndDoubleExecution, region=2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,335 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=86 2023-02-11 09:55:43,335 DEBUG [Listener at localhost/34597] assignment.RegionStateStore(147): Load hbase:meta entry region=304850c3fd2c34006e777e4aba5f33e0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40095,1676109241701, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701, openSeqNum=86 2023-02-11 09:55:43,335 INFO [Listener at localhost/34597] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 09:55:43,335 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676109403335 2023-02-11 09:55:43,335 INFO [Listener at localhost/34597] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676109463335 2023-02-11 09:55:43,335 INFO [Listener at localhost/34597] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 09:55:43,335 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 09:55:43,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2d09da8c5c6e64b92b6df9ca89193aab, daughterA=a8470dda2cae9ecdee54ef2ef9721bcc, daughterB=304850c3fd2c34006e777e4aba5f33e0 in 3.6780 sec 2023-02-11 09:55:43,352 ERROR [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid b979c166-8ceb-4cd3-99a3-1b2b2b1e0014) service to localhost/127.0.0.1:41137 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:55:43,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40095] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:35030 deadline: 1676109403586, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab. is not online on jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,697 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 09:55:43,703 INFO [Listener at localhost/34597] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 09:55:43,705 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 09:55:43,713 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:43,714 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-11 09:55:43,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-11 09:55:43,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:43,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 09:55:43,718 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109343717"}]},"ts":"1676109343717"} 2023-02-11 09:55:43,718 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 09:55:43,720 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-11 09:55:43,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, UNASSIGN}] 2023-02-11 09:55:43,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, UNASSIGN 2023-02-11 09:55:43,721 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=304850c3fd2c34006e777e4aba5f33e0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,722 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109343721"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109343721"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109343721"}]},"ts":"1676109343721"} 2023-02-11 09:55:43,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 304850c3fd2c34006e777e4aba5f33e0, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:43,724 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, UNASSIGN 2023-02-11 09:55:43,724 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:43,725 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109343724"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109343724"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109343724"}]},"ts":"1676109343724"} 2023-02-11 09:55:43,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure a8470dda2cae9ecdee54ef2ef9721bcc, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:43,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 09:55:43,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a8470dda2cae9ecdee54ef2ef9721bcc, disabling compactions & flushes 2023-02-11 09:55:43,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. after waiting 0 ms 2023-02-11 09:55:43,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,875 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90-bottom] to archive 2023-02-11 09:55:43,876 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:43,877 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,878 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,879 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,880 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,881 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,881 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,882 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,883 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05-bottom] to archive 2023-02-11 09:55:43,884 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:43,885 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,886 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,887 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,888 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,889 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,889 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,890 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:55:43,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:43,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc. 2023-02-11 09:55:43,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a8470dda2cae9ecdee54ef2ef9721bcc: Waiting for close lock at 1676109343874Running coprocessor pre-close hooks at 1676109343874Disabling compacts and flushes for region at 1676109343874Disabling writes for close at 1676109343874Writing region close event to WAL at 1676109343891 (+17 ms)Running coprocessor post-close hooks at 1676109343894 (+3 ms)Closed at 1676109343894 2023-02-11 09:55:43,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:43,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 304850c3fd2c34006e777e4aba5f33e0, disabling compactions & flushes 2023-02-11 09:55:43,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. after waiting 0 ms 2023-02-11 09:55:43,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=a8470dda2cae9ecdee54ef2ef9721bcc, regionState=CLOSED 2023-02-11 09:55:43,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109343896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109343896"}]},"ts":"1676109343896"} 2023-02-11 09:55:43,897 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90-top] to archive 2023-02-11 09:55:43,898 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:43,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-11 09:55:43,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure a8470dda2cae9ecdee54ef2ef9721bcc, server=jenkins-hbase4.apache.org,40095,1676109241701 in 173 msec 2023-02-11 09:55:43,899 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a8470dda2cae9ecdee54ef2ef9721bcc, UNASSIGN in 179 msec 2023-02-11 09:55:43,900 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,901 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,903 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,904 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,905 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,905 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,907 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05-top] to archive 2023-02-11 09:55:43,907 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:43,909 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,909 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,910 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,911 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,912 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,913 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,913 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:43,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:55:43,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:43,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0. 2023-02-11 09:55:43,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 304850c3fd2c34006e777e4aba5f33e0: Waiting for close lock at 1676109343896Running coprocessor pre-close hooks at 1676109343896Disabling compacts and flushes for region at 1676109343896Disabling writes for close at 1676109343896Writing region close event to WAL at 1676109343914 (+18 ms)Running coprocessor post-close hooks at 1676109343917 (+3 ms)Closed at 1676109343917 2023-02-11 09:55:43,919 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:43,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=304850c3fd2c34006e777e4aba5f33e0, regionState=CLOSED 2023-02-11 09:55:43,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676109343919"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109343919"}]},"ts":"1676109343919"} 2023-02-11 09:55:43,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-11 09:55:43,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 304850c3fd2c34006e777e4aba5f33e0, server=jenkins-hbase4.apache.org,40095,1676109241701 in 198 msec 2023-02-11 09:55:43,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-11 09:55:43,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=304850c3fd2c34006e777e4aba5f33e0, UNASSIGN in 202 msec 2023-02-11 09:55:43,924 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109343924"}]},"ts":"1676109343924"} 2023-02-11 09:55:43,925 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 09:55:43,928 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-11 09:55:43,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 213 msec 2023-02-11 09:55:44,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 09:55:44,019 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-11 09:55:44,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-11 09:55:44,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:44,022 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:44,022 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:44,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 09:55:44,026 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,026 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:44,026 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:44,028 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/recovered.edits] 2023-02-11 09:55:44,029 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/recovered.edits] 2023-02-11 09:55:44,029 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/recovered.edits] 2023-02-11 09:55:44,032 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/0090de63230e480488b49a157f4d5e90 2023-02-11 09:55:44,032 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/.regioninfo to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/.regioninfo 2023-02-11 09:55:44,032 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/.regioninfo to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/.regioninfo 2023-02-11 09:55:44,034 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd 2023-02-11 09:55:44,035 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,035 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/5ae009ebf178474d93dd666d49a47968 2023-02-11 09:55:44,035 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/0090de63230e480488b49a157f4d5e90.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,036 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/6d80e8b5ef06418793318cbbd4f9a227 2023-02-11 09:55:44,036 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,037 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/3ad9ecc00bcf4eb782f39a4f401c79dd.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,038 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/7dd81a7911244521bf2ca03495cc4264 2023-02-11 09:55:44,038 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,038 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/5ae009ebf178474d93dd666d49a47968.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,039 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/d841a697a87847ba903061e8e5cb66ff 2023-02-11 09:55:44,039 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,039 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/6d80e8b5ef06418793318cbbd4f9a227.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,040 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf1/e2647c8c1fff4de1a2581e4e8bdf3147 2023-02-11 09:55:44,041 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,041 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/7dd81a7911244521bf2ca03495cc4264.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,042 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,042 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/d841a697a87847ba903061e8e5cb66ff.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,043 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,043 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/e2647c8c1fff4de1a2581e4e8bdf3147.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,043 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/22f0483465d6436db54645868ab3d499 2023-02-11 09:55:44,045 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/3ae1c8ec3e7b44fa99537ae1ef522759 2023-02-11 09:55:44,046 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/4847d9bb8ac2417c8f1a6d726f6458b0 2023-02-11 09:55:44,046 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,046 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/22f0483465d6436db54645868ab3d499.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,047 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/66b7b1d6a3464ba0ab8b69e19710ef54 2023-02-11 09:55:44,048 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,048 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/3ae1c8ec3e7b44fa99537ae1ef522759.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,049 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/996013109916492c8436a2bfcfb41ec7 2023-02-11 09:55:44,049 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,049 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/4847d9bb8ac2417c8f1a6d726f6458b0.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,050 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,050 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/c97c25476032483e99937b7bd569ae05 2023-02-11 09:55:44,050 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/66b7b1d6a3464ba0ab8b69e19710ef54.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,051 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,051 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/cf2/ec1fd141bd7c4a898ef345f134fd26b3 2023-02-11 09:55:44,051 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/996013109916492c8436a2bfcfb41ec7.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,052 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,053 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/c97c25476032483e99937b7bd569ae05.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,053 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/304850c3fd2c34006e777e4aba5f33e0/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,054 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/ec1fd141bd7c4a898ef345f134fd26b3.2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,054 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/recovered.edits/85.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab/recovered.edits/85.seqid 2023-02-11 09:55:44,055 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/2d09da8c5c6e64b92b6df9ca89193aab 2023-02-11 09:55:44,056 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/71ecc552c20e45efad8be730f256d4bc to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf1/71ecc552c20e45efad8be730f256d4bc 2023-02-11 09:55:44,056 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/4c3f88fb8f644223a6d389b863c96aa4 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf1/4c3f88fb8f644223a6d389b863c96aa4 2023-02-11 09:55:44,058 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/dafef7510e6240eea62008db7adb94a9 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/cf2/dafef7510e6240eea62008db7adb94a9 2023-02-11 09:55:44,058 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/2df90551701448e0ac9818abe114ebdd to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/cf2/2df90551701448e0ac9818abe114ebdd 2023-02-11 09:55:44,061 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0/recovered.edits/90.seqid 2023-02-11 09:55:44,061 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc/recovered.edits/90.seqid 2023-02-11 09:55:44,061 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/304850c3fd2c34006e777e4aba5f33e0 2023-02-11 09:55:44,061 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testRecoveryAndDoubleExecution/a8470dda2cae9ecdee54ef2ef9721bcc 2023-02-11 09:55:44,061 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 09:55:44,063 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:44,065 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-11 09:55:44,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-11 09:55:44,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:44,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-11 09:55:44,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109344068"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:44,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109344068"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:44,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109344068"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:44,069 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 09:55:44,069 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2d09da8c5c6e64b92b6df9ca89193aab, NAME => 'testRecoveryAndDoubleExecution,,1676109337846.2d09da8c5c6e64b92b6df9ca89193aab.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => a8470dda2cae9ecdee54ef2ef9721bcc, NAME => 'testRecoveryAndDoubleExecution,,1676109339656.a8470dda2cae9ecdee54ef2ef9721bcc.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 304850c3fd2c34006e777e4aba5f33e0, NAME => 'testRecoveryAndDoubleExecution,41,1676109339656.304850c3fd2c34006e777e4aba5f33e0.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 09:55:44,069 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-11 09:55:44,069 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109344069"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:44,070 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-11 09:55:44,072 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 09:55:44,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 52 msec 2023-02-11 09:55:44,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 09:55:44,123 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-11 09:55:44,138 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 369) - Thread LEAK? -, OpenFileDescriptor=835 (was 745) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 105), ProcessCount=174 (was 174), AvailableMemoryMB=2513 (was 2526) 2023-02-11 09:55:44,156 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=174, AvailableMemoryMB=2513 2023-02-11 09:55:44,156 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:44,160 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:55:44,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:55:44,161 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,163 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:55:44,164 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:55:44,165 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,166 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5 empty. 2023-02-11 09:55:44,166 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,166 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 09:55:44,177 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 09:55:44,178 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => f346405310f82d5fa878ab352af09cf5, NAME => 'testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:55:44,185 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:44,185 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing f346405310f82d5fa878ab352af09cf5, disabling compactions & flushes 2023-02-11 09:55:44,185 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,185 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,185 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. after waiting 0 ms 2023-02-11 09:55:44,186 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,186 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,186 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for f346405310f82d5fa878ab352af09cf5: Waiting for close lock at 1676109344185Disabling compacts and flushes for region at 1676109344185Disabling writes for close at 1676109344186 (+1 ms)Writing region close event to WAL at 1676109344186Closed at 1676109344186 2023-02-11 09:55:44,187 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:55:44,188 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109344188"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109344188"}]},"ts":"1676109344188"} 2023-02-11 09:55:44,189 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:55:44,190 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:55:44,190 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109344190"}]},"ts":"1676109344190"} 2023-02-11 09:55:44,191 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-11 09:55:44,194 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:55:44,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:55:44,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:55:44,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:55:44,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:55:44,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f346405310f82d5fa878ab352af09cf5, ASSIGN}] 2023-02-11 09:55:44,197 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f346405310f82d5fa878ab352af09cf5, ASSIGN 2023-02-11 09:55:44,197 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f346405310f82d5fa878ab352af09cf5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:55:44,348 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:55:44,349 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=f346405310f82d5fa878ab352af09cf5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,349 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109344349"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109344349"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109344349"}]},"ts":"1676109344349"} 2023-02-11 09:55:44,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure f346405310f82d5fa878ab352af09cf5, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:44,351 ERROR [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid 75b11f66-72e0-4ff6-b627-e436c09781c9) service to localhost/127.0.0.1:41137 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:55:44,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f346405310f82d5fa878ab352af09cf5, NAME => 'testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:55:44,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:44,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:44,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,506 INFO [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,507 DEBUG [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1 2023-02-11 09:55:44,507 DEBUG [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1 2023-02-11 09:55:44,507 INFO [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f346405310f82d5fa878ab352af09cf5 columnFamilyName cf1 2023-02-11 09:55:44,508 INFO [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] regionserver.HStore(310): Store=f346405310f82d5fa878ab352af09cf5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:44,508 INFO [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,509 DEBUG [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2 2023-02-11 09:55:44,509 DEBUG [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2 2023-02-11 09:55:44,509 INFO [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f346405310f82d5fa878ab352af09cf5 columnFamilyName cf2 2023-02-11 09:55:44,509 INFO [StoreOpener-f346405310f82d5fa878ab352af09cf5-1] regionserver.HStore(310): Store=f346405310f82d5fa878ab352af09cf5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:44,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:44,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:44,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:55:44,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f346405310f82d5fa878ab352af09cf5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73107961, jitterRate=0.0893935114145279}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:44,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f346405310f82d5fa878ab352af09cf5: Running coprocessor pre-open hook at 1676109344505Writing region info on filesystem at 1676109344505Initializing all the Stores at 1676109344505Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109344506 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109344506Cleaning up temporary data from old regions at 1676109344511 (+5 ms)Cleaning up detritus from prior splits at 1676109344511Running coprocessor post-open hooks at 1676109344514 (+3 ms)Region opened successfully at 1676109344514 2023-02-11 09:55:44,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5., pid=127, masterSystemTime=1676109344502 2023-02-11 09:55:44,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,516 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=f346405310f82d5fa878ab352af09cf5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,516 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109344516"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109344516"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109344516"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109344516"}]},"ts":"1676109344516"} 2023-02-11 09:55:44,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-11 09:55:44,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure f346405310f82d5fa878ab352af09cf5, server=jenkins-hbase4.apache.org,40095,1676109241701 in 167 msec 2023-02-11 09:55:44,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-11 09:55:44,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f346405310f82d5fa878ab352af09cf5, ASSIGN in 324 msec 2023-02-11 09:55:44,520 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:55:44,521 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109344520"}]},"ts":"1676109344520"} 2023-02-11 09:55:44,521 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-11 09:55:44,523 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:55:44,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 363 msec 2023-02-11 09:55:44,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:44,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,670 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:44,670 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,670 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 09:55:44,670 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 09:55:44,670 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,670 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,671 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,671 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,671 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,671 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,671 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,672 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:44,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,673 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,673 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:44,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:44,674 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:44,674 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:44,674 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. started... 2023-02-11 09:55:44,675 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f346405310f82d5fa878ab352af09cf5 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,678 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,678 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,678 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:44,678 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,678 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:44,685 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/f44e830ab86e48e6a321807a9c16a611 2023-02-11 09:55:44,704 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/9768f88012904648b1ed2f514833dc4b 2023-02-11 09:55:44,708 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/f44e830ab86e48e6a321807a9c16a611 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f44e830ab86e48e6a321807a9c16a611 2023-02-11 09:55:44,711 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f44e830ab86e48e6a321807a9c16a611, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:44,712 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/9768f88012904648b1ed2f514833dc4b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/9768f88012904648b1ed2f514833dc4b 2023-02-11 09:55:44,715 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/9768f88012904648b1ed2f514833dc4b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:44,715 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 40ms, sequenceid=6, compaction requested=false 2023-02-11 09:55:44,715 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:44,716 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f346405310f82d5fa878ab352af09cf5: 2023-02-11 09:55:44,716 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,716 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:44,716 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:44,716 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,716 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:44,716 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:44,717 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,717 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:44,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 09:55:44,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53933da5[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:44,719 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 09:55:44,719 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,723 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,723 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,723 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:44,724 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,724 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:44,725 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:44,725 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,725 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:44,725 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,725 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:44,725 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:44,726 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:44,726 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,726 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:44,726 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:44,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:55:44,729 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 09:55:44,729 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:44,729 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:44,729 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,729 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:44,730 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:44,730 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,730 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,731 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,731 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:44,731 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:44,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,733 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:44,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:44,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,740 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,740 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,741 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,841 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:44,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:44,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:44,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:44,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,853 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:44,853 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,853 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 09:55:44,853 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 09:55:44,854 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,854 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,855 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,855 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,855 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,855 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,855 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,856 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:44,856 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,857 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:44,857 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:44,857 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:44,858 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:44,858 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,858 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. started... 2023-02-11 09:55:44,858 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,858 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f346405310f82d5fa878ab352af09cf5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:44,860 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,860 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,860 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:44,861 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,861 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:44,873 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/bdf3d55562914a09ac0530c3a62c18ef 2023-02-11 09:55:44,886 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/285715dd027c4a07a99213ea7f2b04fb 2023-02-11 09:55:44,890 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/bdf3d55562914a09ac0530c3a62c18ef as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/bdf3d55562914a09ac0530c3a62c18ef 2023-02-11 09:55:44,893 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/bdf3d55562914a09ac0530c3a62c18ef, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:44,894 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/285715dd027c4a07a99213ea7f2b04fb as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/285715dd027c4a07a99213ea7f2b04fb 2023-02-11 09:55:44,897 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/285715dd027c4a07a99213ea7f2b04fb, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:44,898 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 40ms, sequenceid=19, compaction requested=false 2023-02-11 09:55:44,898 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f346405310f82d5fa878ab352af09cf5: 2023-02-11 09:55:44,898 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:44,898 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:44,899 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:44,899 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,899 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:44,899 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:44,900 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,900 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:44,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:44,901 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,901 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:44,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:44,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:44,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 09:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2802373e[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:44,903 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 09:55:44,903 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,904 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:44,904 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:44,904 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:44,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:44,904 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:44,904 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:44,904 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:44,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,905 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:44,905 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:44,905 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:44,905 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:44,906 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,906 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:44,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:44,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,907 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:44,908 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:44,911 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:55:44,911 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 09:55:44,911 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:44,911 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:44,911 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,911 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:44,913 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,913 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,913 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:44,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,914 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,914 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,915 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:44,916 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,916 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,918 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:44,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:44,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,918 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:44,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:44,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:44,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:44,919 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:44,920 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:44,920 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,019 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:45,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:45,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:45,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:45,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,032 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,032 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,032 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 09:55:45,032 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 09:55:45,032 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,032 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,034 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,034 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,034 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,034 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,034 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,035 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:45,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,037 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,037 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,037 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,037 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,037 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:45,037 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:45,037 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,037 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. started... 2023-02-11 09:55:45,037 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f346405310f82d5fa878ab352af09cf5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:45,038 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,038 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,038 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,038 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,039 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,039 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,046 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/25bac550cf674fbd8a048829cbd53890 2023-02-11 09:55:45,055 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/92b53cdd1b6d42c89e1e7968aa50b84c 2023-02-11 09:55:45,058 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/25bac550cf674fbd8a048829cbd53890 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/25bac550cf674fbd8a048829cbd53890 2023-02-11 09:55:45,061 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/25bac550cf674fbd8a048829cbd53890, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:45,061 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/92b53cdd1b6d42c89e1e7968aa50b84c as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/92b53cdd1b6d42c89e1e7968aa50b84c 2023-02-11 09:55:45,064 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/92b53cdd1b6d42c89e1e7968aa50b84c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:45,065 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 28ms, sequenceid=32, compaction requested=true 2023-02-11 09:55:45,065 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f346405310f82d5fa878ab352af09cf5: 2023-02-11 09:55:45,065 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,065 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:45,065 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:45,065 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,065 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,065 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,067 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,067 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,067 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,067 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 09:55:45,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33cc1737[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:45,069 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 09:55:45,069 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,071 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:45,075 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,075 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,075 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,075 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,075 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,076 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,076 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,075 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,076 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,076 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:45,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:55:45,079 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 09:55:45,079 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:45,079 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:45,079 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,079 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:45,080 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,080 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,080 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,080 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,080 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,081 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,081 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,081 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,081 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,083 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,086 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,086 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,086 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,086 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,087 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,086 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,087 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:45,088 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,088 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,088 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,088 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,088 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,188 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:45,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:45,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:45,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:45,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,200 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,200 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,200 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 09:55:45,200 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 09:55:45,201 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,201 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,202 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,202 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,202 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,202 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,202 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,202 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,203 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,203 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,203 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:45,204 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:45,204 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,204 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. started... 2023-02-11 09:55:45,204 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f346405310f82d5fa878ab352af09cf5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:45,205 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,205 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,205 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,205 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,205 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,205 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,212 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/58488bbc2b6d482591d9ab294657c5c8 2023-02-11 09:55:45,221 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/89d964fdad464bb9a0fed3c4e51371a1 2023-02-11 09:55:45,224 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/58488bbc2b6d482591d9ab294657c5c8 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/58488bbc2b6d482591d9ab294657c5c8 2023-02-11 09:55:45,227 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/58488bbc2b6d482591d9ab294657c5c8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:45,227 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/89d964fdad464bb9a0fed3c4e51371a1 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/89d964fdad464bb9a0fed3c4e51371a1 2023-02-11 09:55:45,230 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/89d964fdad464bb9a0fed3c4e51371a1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:45,231 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 27ms, sequenceid=45, compaction requested=true 2023-02-11 09:55:45,231 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f346405310f82d5fa878ab352af09cf5: 2023-02-11 09:55:45,231 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,232 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,232 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,233 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,233 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 09:55:45,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f5164f6[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:45,234 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 09:55:45,234 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,237 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,237 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:45,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:45,238 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,238 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,238 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,239 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,239 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,239 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,239 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,239 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,239 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,239 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:45,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:55:45,242 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 09:55:45,242 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:45,242 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:45,242 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,242 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:45,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,244 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,243 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,244 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,244 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,246 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,246 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,246 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,252 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:45,252 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,253 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,253 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,253 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,253 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,253 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,253 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,253 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,353 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:45,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:45,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:45,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:45,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,365 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,365 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,365 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 09:55:45,365 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 09:55:45,365 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,365 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,366 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,366 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,367 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,367 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:45,368 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,368 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. started... 2023-02-11 09:55:45,369 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f346405310f82d5fa878ab352af09cf5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:45,369 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,371 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,371 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,372 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,372 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,372 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,376 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/11821c3c93de4ee99c5a811ed85e2d68 2023-02-11 09:55:45,385 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/b5963aa936dd469a8ae03db3875726fb 2023-02-11 09:55:45,388 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/11821c3c93de4ee99c5a811ed85e2d68 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/11821c3c93de4ee99c5a811ed85e2d68 2023-02-11 09:55:45,390 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/11821c3c93de4ee99c5a811ed85e2d68, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:45,391 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/b5963aa936dd469a8ae03db3875726fb as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/b5963aa936dd469a8ae03db3875726fb 2023-02-11 09:55:45,394 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/b5963aa936dd469a8ae03db3875726fb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:45,394 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 25ms, sequenceid=58, compaction requested=true 2023-02-11 09:55:45,394 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f346405310f82d5fa878ab352af09cf5: 2023-02-11 09:55:45,394 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,395 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:45,395 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:45,395 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,395 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,395 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,396 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,396 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,396 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,396 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 09:55:45,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65651aac[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:45,398 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 09:55:45,398 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,399 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:45,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,399 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,399 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,399 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,399 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:45,399 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:45,399 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,400 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,400 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,403 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,403 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,403 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,403 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:45,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:55:45,405 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 09:55:45,406 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:45,406 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:45,406 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,406 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:45,407 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,407 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,407 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,407 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,408 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,408 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,409 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,410 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,410 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,410 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,413 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,413 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,414 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,514 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:45,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:45,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:45,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:45,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,526 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,526 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,526 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 09:55:45,526 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 09:55:45,526 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,526 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,528 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,528 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,528 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,528 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,529 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,529 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,530 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,530 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,530 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,530 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,530 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,531 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,531 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,531 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. started... 2023-02-11 09:55:45,531 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f346405310f82d5fa878ab352af09cf5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:45,534 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,534 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,535 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,535 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,535 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,535 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,541 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/f66ba8e203864b0bbcbf01a01d8082c6 2023-02-11 09:55:45,553 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/58b75dbfe6384a96b8dd538bf2c8ea78 2023-02-11 09:55:45,557 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/f66ba8e203864b0bbcbf01a01d8082c6 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f66ba8e203864b0bbcbf01a01d8082c6 2023-02-11 09:55:45,560 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f66ba8e203864b0bbcbf01a01d8082c6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:45,561 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/58b75dbfe6384a96b8dd538bf2c8ea78 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/58b75dbfe6384a96b8dd538bf2c8ea78 2023-02-11 09:55:45,564 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/58b75dbfe6384a96b8dd538bf2c8ea78, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:45,565 INFO [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 34ms, sequenceid=71, compaction requested=true 2023-02-11 09:55:45,565 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f346405310f82d5fa878ab352af09cf5: 2023-02-11 09:55:45,565 DEBUG [rs(jenkins-hbase4.apache.org,40095,1676109241701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,565 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:45,565 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:45,565 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,565 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 09:55:45,565 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 09:55:45,567 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,567 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,567 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,567 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:45,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 09:55:45,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ce77af9[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:45,568 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 09:55:45,569 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,574 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:45,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,574 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,574 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,574 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 09:55:45,575 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,575 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:45,575 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 09:55:45,575 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:45,574 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,576 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 09:55:45,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40095,1676109241701': 2023-02-11 09:55:45,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 09:55:45,579 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 09:55:45,579 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:45,579 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:45,579 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,579 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:45,580 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,580 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:45,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,580 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,581 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,581 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,582 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:45,583 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,583 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,584 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 09:55:45,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,586 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:45,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:45,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:45,587 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:45,587 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,586 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:45,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:45,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,587 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:45,587 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:45,587 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:45,587 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,587 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 09:55:45,687 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:45,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:45,695 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,696 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=f346405310f82d5fa878ab352af09cf5, daughterA=da22b7765fa4d44f848e135ce842d483, daughterB=b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:45,697 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=f346405310f82d5fa878ab352af09cf5, daughterA=da22b7765fa4d44f848e135ce842d483, daughterB=b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:45,697 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=f346405310f82d5fa878ab352af09cf5, daughterA=da22b7765fa4d44f848e135ce842d483, daughterB=b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:45,697 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=f346405310f82d5fa878ab352af09cf5, daughterA=da22b7765fa4d44f848e135ce842d483, daughterB=b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:45,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f346405310f82d5fa878ab352af09cf5, UNASSIGN}] 2023-02-11 09:55:45,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f346405310f82d5fa878ab352af09cf5, UNASSIGN 2023-02-11 09:55:45,699 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=f346405310f82d5fa878ab352af09cf5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:45,699 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109345699"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109345699"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109345699"}]},"ts":"1676109345699"} 2023-02-11 09:55:45,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure f346405310f82d5fa878ab352af09cf5, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:45,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f346405310f82d5fa878ab352af09cf5, disabling compactions & flushes 2023-02-11 09:55:45,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. after waiting 0 ms 2023-02-11 09:55:45,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing f346405310f82d5fa878ab352af09cf5 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 09:55:45,861 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/af9bb59152bd41ba9fe433c7d6cac0fa 2023-02-11 09:55:45,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/452ea98fc50d464ca59b2e0bf698b0ba 2023-02-11 09:55:45,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf1/af9bb59152bd41ba9fe433c7d6cac0fa as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/af9bb59152bd41ba9fe433c7d6cac0fa 2023-02-11 09:55:45,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/af9bb59152bd41ba9fe433c7d6cac0fa, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:45,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.tmp/cf2/452ea98fc50d464ca59b2e0bf698b0ba as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/452ea98fc50d464ca59b2e0bf698b0ba 2023-02-11 09:55:45,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/452ea98fc50d464ca59b2e0bf698b0ba, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:45,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 31ms, sequenceid=82, compaction requested=true 2023-02-11 09:55:45,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 09:55:45,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:45,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. 2023-02-11 09:55:45,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f346405310f82d5fa878ab352af09cf5: Waiting for close lock at 1676109345852Running coprocessor pre-close hooks at 1676109345852Disabling compacts and flushes for region at 1676109345852Disabling writes for close at 1676109345852Obtaining lock to block concurrent updates at 1676109345852Preparing flush snapshotting stores in f346405310f82d5fa878ab352af09cf5 at 1676109345852Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676109345852Flushing stores of testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. at 1676109345853 (+1 ms)Flushing f346405310f82d5fa878ab352af09cf5/cf1: creating writer at 1676109345854 (+1 ms)Flushing f346405310f82d5fa878ab352af09cf5/cf1: appending metadata at 1676109345855 (+1 ms)Flushing f346405310f82d5fa878ab352af09cf5/cf1: closing flushed file at 1676109345855Flushing f346405310f82d5fa878ab352af09cf5/cf2: creating writer at 1676109345864 (+9 ms)Flushing f346405310f82d5fa878ab352af09cf5/cf2: appending metadata at 1676109345865 (+1 ms)Flushing f346405310f82d5fa878ab352af09cf5/cf2: closing flushed file at 1676109345866 (+1 ms)Flushing f346405310f82d5fa878ab352af09cf5/cf1: reopening flushed file at 1676109345877 (+11 ms)Flushing f346405310f82d5fa878ab352af09cf5/cf2: reopening flushed file at 1676109345880 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f346405310f82d5fa878ab352af09cf5 in 31ms, sequenceid=82, compaction requested=true at 1676109345883 (+3 ms)Writing region close event to WAL at 1676109345885 (+2 ms)Running coprocessor post-close hooks at 1676109345888 (+3 ms)Closed at 1676109345888 2023-02-11 09:55:45,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,889 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=f346405310f82d5fa878ab352af09cf5, regionState=CLOSED 2023-02-11 09:55:45,889 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109345889"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109345889"}]},"ts":"1676109345889"} 2023-02-11 09:55:45,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-11 09:55:45,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure f346405310f82d5fa878ab352af09cf5, server=jenkins-hbase4.apache.org,40095,1676109241701 in 190 msec 2023-02-11 09:55:45,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-11 09:55:45,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f346405310f82d5fa878ab352af09cf5, UNASSIGN in 194 msec 2023-02-11 09:55:45,901 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=f346405310f82d5fa878ab352af09cf5, threads=14 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/285715dd027c4a07a99213ea7f2b04fb for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/452ea98fc50d464ca59b2e0bf698b0ba for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/58b75dbfe6384a96b8dd538bf2c8ea78 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/89d964fdad464bb9a0fed3c4e51371a1 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/92b53cdd1b6d42c89e1e7968aa50b84c for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/9768f88012904648b1ed2f514833dc4b for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/b5963aa936dd469a8ae03db3875726fb for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/11821c3c93de4ee99c5a811ed85e2d68 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,902 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/25bac550cf674fbd8a048829cbd53890 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,903 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/58488bbc2b6d482591d9ab294657c5c8 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,903 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/af9bb59152bd41ba9fe433c7d6cac0fa for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,903 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/bdf3d55562914a09ac0530c3a62c18ef for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,903 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f44e830ab86e48e6a321807a9c16a611 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,903 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f66ba8e203864b0bbcbf01a01d8082c6 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,906 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/89d964fdad464bb9a0fed3c4e51371a1, top=false 2023-02-11 09:55:45,906 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/58488bbc2b6d482591d9ab294657c5c8, top=false 2023-02-11 09:55:45,912 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-89d964fdad464bb9a0fed3c4e51371a1 for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,914 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/b5963aa936dd469a8ae03db3875726fb, top=false 2023-02-11 09:55:45,914 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/9768f88012904648b1ed2f514833dc4b, top=false 2023-02-11 09:55:45,917 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/58b75dbfe6384a96b8dd538bf2c8ea78, top=false 2023-02-11 09:55:45,917 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/452ea98fc50d464ca59b2e0bf698b0ba, top=false 2023-02-11 09:55:45,917 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f66ba8e203864b0bbcbf01a01d8082c6, top=false 2023-02-11 09:55:45,919 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/bdf3d55562914a09ac0530c3a62c18ef, top=false 2023-02-11 09:55:45,919 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/285715dd027c4a07a99213ea7f2b04fb, top=false 2023-02-11 09:55:45,919 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/11821c3c93de4ee99c5a811ed85e2d68, top=false 2023-02-11 09:55:45,921 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/89d964fdad464bb9a0fed3c4e51371a1 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,921 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/af9bb59152bd41ba9fe433c7d6cac0fa, top=false 2023-02-11 09:55:45,922 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/25bac550cf674fbd8a048829cbd53890, top=false 2023-02-11 09:55:45,922 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/92b53cdd1b6d42c89e1e7968aa50b84c, top=false 2023-02-11 09:55:45,922 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f44e830ab86e48e6a321807a9c16a611, top=false 2023-02-11 09:55:45,927 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-285715dd027c4a07a99213ea7f2b04fb for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,931 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/285715dd027c4a07a99213ea7f2b04fb for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,932 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f44e830ab86e48e6a321807a9c16a611 for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,932 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58488bbc2b6d482591d9ab294657c5c8 for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,937 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-af9bb59152bd41ba9fe433c7d6cac0fa for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,938 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-11821c3c93de4ee99c5a811ed85e2d68 for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,939 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/58488bbc2b6d482591d9ab294657c5c8 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,939 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f44e830ab86e48e6a321807a9c16a611 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,940 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-452ea98fc50d464ca59b2e0bf698b0ba for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,940 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-b5963aa936dd469a8ae03db3875726fb for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,940 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58b75dbfe6384a96b8dd538bf2c8ea78 for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,944 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/11821c3c93de4ee99c5a811ed85e2d68 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,944 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/b5963aa936dd469a8ae03db3875726fb for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,944 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/af9bb59152bd41ba9fe433c7d6cac0fa for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,948 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/452ea98fc50d464ca59b2e0bf698b0ba for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,949 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-25bac550cf674fbd8a048829cbd53890 for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,949 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-bdf3d55562914a09ac0530c3a62c18ef for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,949 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-92b53cdd1b6d42c89e1e7968aa50b84c for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,949 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-9768f88012904648b1ed2f514833dc4b for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,949 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/58b75dbfe6384a96b8dd538bf2c8ea78 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,951 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/.splits/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f66ba8e203864b0bbcbf01a01d8082c6 for child: da22b7765fa4d44f848e135ce842d483, parent: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,952 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/25bac550cf674fbd8a048829cbd53890 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,953 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/bdf3d55562914a09ac0530c3a62c18ef for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,953 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/9768f88012904648b1ed2f514833dc4b for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,954 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f66ba8e203864b0bbcbf01a01d8082c6 for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,956 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/92b53cdd1b6d42c89e1e7968aa50b84c for region: f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:45,956 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region f346405310f82d5fa878ab352af09cf5 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-11 09:55:45,981 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:55:45,983 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:55:45,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676109345984"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676109345984"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676109345984"}]},"ts":"1676109345984"} 2023-02-11 09:55:45,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109345984"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109345984"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109345984"}]},"ts":"1676109345984"} 2023-02-11 09:55:45,985 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109345984"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109345984"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109345984"}]},"ts":"1676109345984"} 2023-02-11 09:55:45,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=da22b7765fa4d44f848e135ce842d483, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b95746b8afb59a76d827f4dbd4dc4bf0, ASSIGN}] 2023-02-11 09:55:45,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b95746b8afb59a76d827f4dbd4dc4bf0, ASSIGN 2023-02-11 09:55:45,989 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b95746b8afb59a76d827f4dbd4dc4bf0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:55:45,989 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=da22b7765fa4d44f848e135ce842d483, ASSIGN 2023-02-11 09:55:45,990 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=da22b7765fa4d44f848e135ce842d483, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40095,1676109241701; forceNewPlan=false, retain=false 2023-02-11 09:55:46,139 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 09:55:46,141 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b95746b8afb59a76d827f4dbd4dc4bf0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:46,141 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346141"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109346141"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109346141"}]},"ts":"1676109346141"} 2023-02-11 09:55:46,142 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:55:46,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure b95746b8afb59a76d827f4dbd4dc4bf0, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:46,145 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=da22b7765fa4d44f848e135ce842d483, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:46,146 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346145"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109346145"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109346145"}]},"ts":"1676109346145"} 2023-02-11 09:55:46,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure da22b7765fa4d44f848e135ce842d483, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:46,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. 2023-02-11 09:55:46,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b95746b8afb59a76d827f4dbd4dc4bf0, NAME => 'testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.', STARTKEY => '71', ENDKEY => ''} 2023-02-11 09:55:46,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:46,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:46,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,299 INFO [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,300 DEBUG [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/cf1 2023-02-11 09:55:46,300 DEBUG [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/cf1 2023-02-11 09:55:46,301 INFO [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b95746b8afb59a76d827f4dbd4dc4bf0 columnFamilyName cf1 2023-02-11 09:55:46,301 INFO [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] regionserver.HStore(310): Store=b95746b8afb59a76d827f4dbd4dc4bf0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:46,301 INFO [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,327 DEBUG [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/cf2 2023-02-11 09:55:46,327 DEBUG [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/cf2 2023-02-11 09:55:46,328 INFO [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b95746b8afb59a76d827f4dbd4dc4bf0 columnFamilyName cf2 2023-02-11 09:55:46,328 INFO [StoreOpener-b95746b8afb59a76d827f4dbd4dc4bf0-1] regionserver.HStore(310): Store=b95746b8afb59a76d827f4dbd4dc4bf0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:46,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:46,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b95746b8afb59a76d827f4dbd4dc4bf0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69472404, jitterRate=0.03521949052810669}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:46,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b95746b8afb59a76d827f4dbd4dc4bf0: Running coprocessor pre-open hook at 1676109346298Writing region info on filesystem at 1676109346298Initializing all the Stores at 1676109346298Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109346298Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109346299 (+1 ms)Cleaning up temporary data from old regions at 1676109346330 (+31 ms)Cleaning up detritus from prior splits at 1676109346330Running coprocessor post-open hooks at 1676109346332 (+2 ms)Region opened successfully at 1676109346332 2023-02-11 09:55:46,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0., pid=133, masterSystemTime=1676109346295 2023-02-11 09:55:46,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. 2023-02-11 09:55:46,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. 2023-02-11 09:55:46,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => da22b7765fa4d44f848e135ce842d483, NAME => 'testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.', STARTKEY => '', ENDKEY => '71'} 2023-02-11 09:55:46,334 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b95746b8afb59a76d827f4dbd4dc4bf0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:46,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:46,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346334"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109346334"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109346334"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109346334"}]},"ts":"1676109346334"} 2023-02-11 09:55:46,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:46,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,336 INFO [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,337 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1 2023-02-11 09:55:46,337 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1 2023-02-11 09:55:46,337 INFO [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da22b7765fa4d44f848e135ce842d483 columnFamilyName cf1 2023-02-11 09:55:46,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-11 09:55:46,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure b95746b8afb59a76d827f4dbd4dc4bf0, server=jenkins-hbase4.apache.org,40095,1676109241701 in 192 msec 2023-02-11 09:55:46,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b95746b8afb59a76d827f4dbd4dc4bf0, ASSIGN in 350 msec 2023-02-11 09:55:46,342 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-11821c3c93de4ee99c5a811ed85e2d68 2023-02-11 09:55:46,345 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-25bac550cf674fbd8a048829cbd53890 2023-02-11 09:55:46,347 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58488bbc2b6d482591d9ab294657c5c8 2023-02-11 09:55:46,350 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-af9bb59152bd41ba9fe433c7d6cac0fa 2023-02-11 09:55:46,352 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-bdf3d55562914a09ac0530c3a62c18ef 2023-02-11 09:55:46,354 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f44e830ab86e48e6a321807a9c16a611 2023-02-11 09:55:46,356 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f66ba8e203864b0bbcbf01a01d8082c6 2023-02-11 09:55:46,356 INFO [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(310): Store=da22b7765fa4d44f848e135ce842d483/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:46,356 INFO [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,357 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2 2023-02-11 09:55:46,357 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2 2023-02-11 09:55:46,357 INFO [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da22b7765fa4d44f848e135ce842d483 columnFamilyName cf2 2023-02-11 09:55:46,361 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-285715dd027c4a07a99213ea7f2b04fb 2023-02-11 09:55:46,363 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-452ea98fc50d464ca59b2e0bf698b0ba 2023-02-11 09:55:46,365 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58b75dbfe6384a96b8dd538bf2c8ea78 2023-02-11 09:55:46,367 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-89d964fdad464bb9a0fed3c4e51371a1 2023-02-11 09:55:46,369 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-92b53cdd1b6d42c89e1e7968aa50b84c 2023-02-11 09:55:46,372 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-9768f88012904648b1ed2f514833dc4b 2023-02-11 09:55:46,374 DEBUG [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-b5963aa936dd469a8ae03db3875726fb 2023-02-11 09:55:46,374 INFO [StoreOpener-da22b7765fa4d44f848e135ce842d483-1] regionserver.HStore(310): Store=da22b7765fa4d44f848e135ce842d483/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:46,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:46,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened da22b7765fa4d44f848e135ce842d483; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68501879, jitterRate=0.020757541060447693}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:46,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for da22b7765fa4d44f848e135ce842d483: Running coprocessor pre-open hook at 1676109346335Writing region info on filesystem at 1676109346335Initializing all the Stores at 1676109346335Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109346335Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109346336 (+1 ms)Cleaning up temporary data from old regions at 1676109346376 (+40 ms)Cleaning up detritus from prior splits at 1676109346376Running coprocessor post-open hooks at 1676109346378 (+2 ms)Region opened successfully at 1676109346378 2023-02-11 09:55:46,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483., pid=134, masterSystemTime=1676109346295 2023-02-11 09:55:46,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:46,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:55:46,380 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:46,380 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:46,380 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 09:55:46,380 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 09:55:46,380 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1912): da22b7765fa4d44f848e135ce842d483/cf1 is initiating minor compaction (all files) 2023-02-11 09:55:46,380 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1912): da22b7765fa4d44f848e135ce842d483/cf2 is initiating minor compaction (all files) 2023-02-11 09:55:46,380 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of da22b7765fa4d44f848e135ce842d483/cf1 in testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,380 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2258): Starting compaction of da22b7765fa4d44f848e135ce842d483/cf2 in testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,380 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f44e830ab86e48e6a321807a9c16a611, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-bdf3d55562914a09ac0530c3a62c18ef, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-25bac550cf674fbd8a048829cbd53890, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58488bbc2b6d482591d9ab294657c5c8, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-11821c3c93de4ee99c5a811ed85e2d68, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f66ba8e203864b0bbcbf01a01d8082c6, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-af9bb59152bd41ba9fe433c7d6cac0fa] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/.tmp, totalSize=34.9 K 2023-02-11 09:55:46,380 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-9768f88012904648b1ed2f514833dc4b, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-285715dd027c4a07a99213ea7f2b04fb, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-92b53cdd1b6d42c89e1e7968aa50b84c, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-89d964fdad464bb9a0fed3c4e51371a1, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-b5963aa936dd469a8ae03db3875726fb, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58b75dbfe6384a96b8dd538bf2c8ea78, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-452ea98fc50d464ca59b2e0bf698b0ba] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/.tmp, totalSize=34.9 K 2023-02-11 09:55:46,381 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f44e830ab86e48e6a321807a9c16a611, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109344664 2023-02-11 09:55:46,381 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-9768f88012904648b1ed2f514833dc4b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109344664 2023-02-11 09:55:46,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,381 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-bdf3d55562914a09ac0530c3a62c18ef, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109344842 2023-02-11 09:55:46,381 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-285715dd027c4a07a99213ea7f2b04fb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109344842 2023-02-11 09:55:46,382 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=da22b7765fa4d44f848e135ce842d483, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:46,382 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-25bac550cf674fbd8a048829cbd53890, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109345021 2023-02-11 09:55:46,382 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346382"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109346382"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109346382"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109346382"}]},"ts":"1676109346382"} 2023-02-11 09:55:46,382 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-92b53cdd1b6d42c89e1e7968aa50b84c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109345021 2023-02-11 09:55:46,382 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58488bbc2b6d482591d9ab294657c5c8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109345189 2023-02-11 09:55:46,382 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-89d964fdad464bb9a0fed3c4e51371a1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109345189 2023-02-11 09:55:46,382 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-11821c3c93de4ee99c5a811ed85e2d68, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109345354 2023-02-11 09:55:46,383 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-b5963aa936dd469a8ae03db3875726fb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109345354 2023-02-11 09:55:46,383 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f66ba8e203864b0bbcbf01a01d8082c6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109345515 2023-02-11 09:55:46,383 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58b75dbfe6384a96b8dd538bf2c8ea78, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109345515 2023-02-11 09:55:46,383 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-af9bb59152bd41ba9fe433c7d6cac0fa, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109345688 2023-02-11 09:55:46,383 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-452ea98fc50d464ca59b2e0bf698b0ba, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109345688 2023-02-11 09:55:46,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-11 09:55:46,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure da22b7765fa4d44f848e135ce842d483, server=jenkins-hbase4.apache.org,40095,1676109241701 in 236 msec 2023-02-11 09:55:46,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-11 09:55:46,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=da22b7765fa4d44f848e135ce842d483, ASSIGN in 400 msec 2023-02-11 09:55:46,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=f346405310f82d5fa878ab352af09cf5, daughterA=da22b7765fa4d44f848e135ce842d483, daughterB=b95746b8afb59a76d827f4dbd4dc4bf0 in 693 msec 2023-02-11 09:55:46,400 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] throttle.PressureAwareThroughputController(145): da22b7765fa4d44f848e135ce842d483#cf2#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:46,400 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] throttle.PressureAwareThroughputController(145): da22b7765fa4d44f848e135ce842d483#cf1#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:46,409 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/.tmp/cf2/8c516bb596154245aaf21e0679b10bc6 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/8c516bb596154245aaf21e0679b10bc6 2023-02-11 09:55:46,409 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/.tmp/cf1/0698b4e1b45244f7bf89826f263f1eb1 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/0698b4e1b45244f7bf89826f263f1eb1 2023-02-11 09:55:46,413 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in da22b7765fa4d44f848e135ce842d483/cf2 of da22b7765fa4d44f848e135ce842d483 into 8c516bb596154245aaf21e0679b10bc6(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:46,413 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in da22b7765fa4d44f848e135ce842d483/cf1 of da22b7765fa4d44f848e135ce842d483 into 0698b4e1b45244f7bf89826f263f1eb1(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:46,413 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for da22b7765fa4d44f848e135ce842d483: 2023-02-11 09:55:46,413 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for da22b7765fa4d44f848e135ce842d483: 2023-02-11 09:55:46,413 INFO [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483., storeName=da22b7765fa4d44f848e135ce842d483/cf2, priority=9, startTime=1676109346380; duration=0sec 2023-02-11 09:55:46,413 INFO [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483., storeName=da22b7765fa4d44f848e135ce842d483/cf1, priority=9, startTime=1676109346379; duration=0sec 2023-02-11 09:55:46,413 DEBUG [RS:0;jenkins-hbase4:40095-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:46,414 DEBUG [RS:0;jenkins-hbase4:40095-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:46,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40095] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:35030 deadline: 1676109406447, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5. is not online on jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:46,556 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-11 09:55:46,559 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:46,560 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 09:55:46,564 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109346564"}]},"ts":"1676109346564"} 2023-02-11 09:55:46,565 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-11 09:55:46,567 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-11 09:55:46,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=da22b7765fa4d44f848e135ce842d483, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b95746b8afb59a76d827f4dbd4dc4bf0, UNASSIGN}] 2023-02-11 09:55:46,568 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b95746b8afb59a76d827f4dbd4dc4bf0, UNASSIGN 2023-02-11 09:55:46,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b95746b8afb59a76d827f4dbd4dc4bf0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:46,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346569"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109346569"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109346569"}]},"ts":"1676109346569"} 2023-02-11 09:55:46,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure b95746b8afb59a76d827f4dbd4dc4bf0, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:46,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=da22b7765fa4d44f848e135ce842d483, UNASSIGN 2023-02-11 09:55:46,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=da22b7765fa4d44f848e135ce842d483, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:46,572 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346572"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109346572"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109346572"}]},"ts":"1676109346572"} 2023-02-11 09:55:46,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure da22b7765fa4d44f848e135ce842d483, server=jenkins-hbase4.apache.org,40095,1676109241701}] 2023-02-11 09:55:46,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 09:55:46,721 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing da22b7765fa4d44f848e135ce842d483, disabling compactions & flushes 2023-02-11 09:55:46,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. after waiting 0 ms 2023-02-11 09:55:46,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,722 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f44e830ab86e48e6a321807a9c16a611, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-bdf3d55562914a09ac0530c3a62c18ef, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-25bac550cf674fbd8a048829cbd53890, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58488bbc2b6d482591d9ab294657c5c8, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-11821c3c93de4ee99c5a811ed85e2d68, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f66ba8e203864b0bbcbf01a01d8082c6, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-af9bb59152bd41ba9fe433c7d6cac0fa] to archive 2023-02-11 09:55:46,723 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:46,724 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f44e830ab86e48e6a321807a9c16a611 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f44e830ab86e48e6a321807a9c16a611 2023-02-11 09:55:46,725 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-bdf3d55562914a09ac0530c3a62c18ef to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-bdf3d55562914a09ac0530c3a62c18ef 2023-02-11 09:55:46,726 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-25bac550cf674fbd8a048829cbd53890 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-25bac550cf674fbd8a048829cbd53890 2023-02-11 09:55:46,728 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58488bbc2b6d482591d9ab294657c5c8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58488bbc2b6d482591d9ab294657c5c8 2023-02-11 09:55:46,729 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-11821c3c93de4ee99c5a811ed85e2d68 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-11821c3c93de4ee99c5a811ed85e2d68 2023-02-11 09:55:46,730 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f66ba8e203864b0bbcbf01a01d8082c6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-f66ba8e203864b0bbcbf01a01d8082c6 2023-02-11 09:55:46,731 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-af9bb59152bd41ba9fe433c7d6cac0fa to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-af9bb59152bd41ba9fe433c7d6cac0fa 2023-02-11 09:55:46,732 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-9768f88012904648b1ed2f514833dc4b, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-285715dd027c4a07a99213ea7f2b04fb, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-92b53cdd1b6d42c89e1e7968aa50b84c, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-89d964fdad464bb9a0fed3c4e51371a1, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-b5963aa936dd469a8ae03db3875726fb, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58b75dbfe6384a96b8dd538bf2c8ea78, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-452ea98fc50d464ca59b2e0bf698b0ba] to archive 2023-02-11 09:55:46,732 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:46,733 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-9768f88012904648b1ed2f514833dc4b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-9768f88012904648b1ed2f514833dc4b 2023-02-11 09:55:46,734 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-285715dd027c4a07a99213ea7f2b04fb to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-285715dd027c4a07a99213ea7f2b04fb 2023-02-11 09:55:46,735 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-92b53cdd1b6d42c89e1e7968aa50b84c to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-92b53cdd1b6d42c89e1e7968aa50b84c 2023-02-11 09:55:46,736 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-89d964fdad464bb9a0fed3c4e51371a1 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-89d964fdad464bb9a0fed3c4e51371a1 2023-02-11 09:55:46,737 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-b5963aa936dd469a8ae03db3875726fb to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-b5963aa936dd469a8ae03db3875726fb 2023-02-11 09:55:46,738 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58b75dbfe6384a96b8dd538bf2c8ea78 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-58b75dbfe6384a96b8dd538bf2c8ea78 2023-02-11 09:55:46,739 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-452ea98fc50d464ca59b2e0bf698b0ba to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/testSplitTableRegionEmptyDaughter=f346405310f82d5fa878ab352af09cf5-452ea98fc50d464ca59b2e0bf698b0ba 2023-02-11 09:55:46,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:55:46,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:46,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483. 2023-02-11 09:55:46,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for da22b7765fa4d44f848e135ce842d483: Waiting for close lock at 1676109346722Running coprocessor pre-close hooks at 1676109346722Disabling compacts and flushes for region at 1676109346722Disabling writes for close at 1676109346722Writing region close event to WAL at 1676109346740 (+18 ms)Running coprocessor post-close hooks at 1676109346742 (+2 ms)Closed at 1676109346742 2023-02-11 09:55:46,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b95746b8afb59a76d827f4dbd4dc4bf0, disabling compactions & flushes 2023-02-11 09:55:46,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. 2023-02-11 09:55:46,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. 2023-02-11 09:55:46,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. after waiting 0 ms 2023-02-11 09:55:46,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. 2023-02-11 09:55:46,745 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=da22b7765fa4d44f848e135ce842d483, regionState=CLOSED 2023-02-11 09:55:46,745 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346745"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109346745"}]},"ts":"1676109346745"} 2023-02-11 09:55:46,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-11 09:55:46,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure da22b7765fa4d44f848e135ce842d483, server=jenkins-hbase4.apache.org,40095,1676109241701 in 173 msec 2023-02-11 09:55:46,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-11 09:55:46,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:46,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0. 2023-02-11 09:55:46,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b95746b8afb59a76d827f4dbd4dc4bf0: Waiting for close lock at 1676109346744Running coprocessor pre-close hooks at 1676109346744Disabling compacts and flushes for region at 1676109346744Disabling writes for close at 1676109346745 (+1 ms)Writing region close event to WAL at 1676109346745Running coprocessor post-close hooks at 1676109346748 (+3 ms)Closed at 1676109346748 2023-02-11 09:55:46,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=da22b7765fa4d44f848e135ce842d483, UNASSIGN in 179 msec 2023-02-11 09:55:46,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,749 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b95746b8afb59a76d827f4dbd4dc4bf0, regionState=CLOSED 2023-02-11 09:55:46,749 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676109346749"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109346749"}]},"ts":"1676109346749"} 2023-02-11 09:55:46,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-11 09:55:46,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure b95746b8afb59a76d827f4dbd4dc4bf0, server=jenkins-hbase4.apache.org,40095,1676109241701 in 180 msec 2023-02-11 09:55:46,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-11 09:55:46,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b95746b8afb59a76d827f4dbd4dc4bf0, UNASSIGN in 183 msec 2023-02-11 09:55:46,753 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109346753"}]},"ts":"1676109346753"} 2023-02-11 09:55:46,753 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-11 09:55:46,755 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-11 09:55:46,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 194 msec 2023-02-11 09:55:46,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 09:55:46,865 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-11 09:55:46,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,868 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,869 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 09:55:46,872 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:46,872 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,872 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,874 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/recovered.edits] 2023-02-11 09:55:46,875 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/recovered.edits] 2023-02-11 09:55:46,875 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/recovered.edits] 2023-02-11 09:55:46,877 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/11821c3c93de4ee99c5a811ed85e2d68 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/11821c3c93de4ee99c5a811ed85e2d68 2023-02-11 09:55:46,877 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/0698b4e1b45244f7bf89826f263f1eb1 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf1/0698b4e1b45244f7bf89826f263f1eb1 2023-02-11 09:55:46,878 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/25bac550cf674fbd8a048829cbd53890 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/25bac550cf674fbd8a048829cbd53890 2023-02-11 09:55:46,879 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/recovered.edits/88.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0/recovered.edits/88.seqid 2023-02-11 09:55:46,880 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/58488bbc2b6d482591d9ab294657c5c8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/58488bbc2b6d482591d9ab294657c5c8 2023-02-11 09:55:46,880 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/b95746b8afb59a76d827f4dbd4dc4bf0 2023-02-11 09:55:46,880 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/8c516bb596154245aaf21e0679b10bc6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/cf2/8c516bb596154245aaf21e0679b10bc6 2023-02-11 09:55:46,881 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/af9bb59152bd41ba9fe433c7d6cac0fa to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/af9bb59152bd41ba9fe433c7d6cac0fa 2023-02-11 09:55:46,882 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/bdf3d55562914a09ac0530c3a62c18ef to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/bdf3d55562914a09ac0530c3a62c18ef 2023-02-11 09:55:46,882 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483/recovered.edits/90.seqid 2023-02-11 09:55:46,883 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/da22b7765fa4d44f848e135ce842d483 2023-02-11 09:55:46,883 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f44e830ab86e48e6a321807a9c16a611 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f44e830ab86e48e6a321807a9c16a611 2023-02-11 09:55:46,884 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f66ba8e203864b0bbcbf01a01d8082c6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf1/f66ba8e203864b0bbcbf01a01d8082c6 2023-02-11 09:55:46,886 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/285715dd027c4a07a99213ea7f2b04fb to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/285715dd027c4a07a99213ea7f2b04fb 2023-02-11 09:55:46,887 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/452ea98fc50d464ca59b2e0bf698b0ba to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/452ea98fc50d464ca59b2e0bf698b0ba 2023-02-11 09:55:46,888 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/58b75dbfe6384a96b8dd538bf2c8ea78 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/58b75dbfe6384a96b8dd538bf2c8ea78 2023-02-11 09:55:46,889 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/89d964fdad464bb9a0fed3c4e51371a1 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/89d964fdad464bb9a0fed3c4e51371a1 2023-02-11 09:55:46,890 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/92b53cdd1b6d42c89e1e7968aa50b84c to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/92b53cdd1b6d42c89e1e7968aa50b84c 2023-02-11 09:55:46,891 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/9768f88012904648b1ed2f514833dc4b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/9768f88012904648b1ed2f514833dc4b 2023-02-11 09:55:46,892 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/b5963aa936dd469a8ae03db3875726fb to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/cf2/b5963aa936dd469a8ae03db3875726fb 2023-02-11 09:55:46,893 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/recovered.edits/85.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5/recovered.edits/85.seqid 2023-02-11 09:55:46,894 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegionEmptyDaughter/f346405310f82d5fa878ab352af09cf5 2023-02-11 09:55:46,894 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 09:55:46,896 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,897 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-11 09:55:46,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-11 09:55:46,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-11 09:55:46,900 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109346899"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:46,900 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109346899"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:46,900 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109346899"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:46,901 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 09:55:46,901 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f346405310f82d5fa878ab352af09cf5, NAME => 'testSplitTableRegionEmptyDaughter,,1676109344160.f346405310f82d5fa878ab352af09cf5.', STARTKEY => '', ENDKEY => ''}, {ENCODED => da22b7765fa4d44f848e135ce842d483, NAME => 'testSplitTableRegionEmptyDaughter,,1676109345695.da22b7765fa4d44f848e135ce842d483.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => b95746b8afb59a76d827f4dbd4dc4bf0, NAME => 'testSplitTableRegionEmptyDaughter,71,1676109345695.b95746b8afb59a76d827f4dbd4dc4bf0.', STARTKEY => '71', ENDKEY => ''}] 2023-02-11 09:55:46,901 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-11 09:55:46,901 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109346901"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:46,902 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-11 09:55:46,904 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 09:55:46,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 38 msec 2023-02-11 09:55:46,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 09:55:46,970 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-11 09:55:46,984 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 376), OpenFileDescriptor=833 (was 835), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 104), ProcessCount=174 (was 174), AvailableMemoryMB=2462 (was 2513) 2023-02-11 09:55:46,999 INFO [Listener at localhost/34597] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=174, AvailableMemoryMB=2461 2023-02-11 09:55:46,999 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:47,002 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 09:55:47,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 09:55:47,003 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-11 09:55:47,005 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 09:55:47,006 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 09:55:47,007 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,008 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95 empty. 2023-02-11 09:55:47,008 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,008 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 09:55:47,027 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-11 09:55:47,029 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 86352aa42570636b3aa8a48d5d7ced95, NAME => 'testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp 2023-02-11 09:55:47,038 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:47,038 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 86352aa42570636b3aa8a48d5d7ced95, disabling compactions & flushes 2023-02-11 09:55:47,038 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,038 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,038 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. after waiting 0 ms 2023-02-11 09:55:47,038 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,038 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,038 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 86352aa42570636b3aa8a48d5d7ced95: Waiting for close lock at 1676109347038Disabling compacts and flushes for region at 1676109347038Disabling writes for close at 1676109347038Writing region close event to WAL at 1676109347038Closed at 1676109347038 2023-02-11 09:55:47,040 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 09:55:47,041 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109347041"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109347041"}]},"ts":"1676109347041"} 2023-02-11 09:55:47,042 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 09:55:47,043 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 09:55:47,043 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109347043"}]},"ts":"1676109347043"} 2023-02-11 09:55:47,044 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-11 09:55:47,047 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 09:55:47,048 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 09:55:47,048 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 09:55:47,048 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 09:55:47,048 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 09:55:47,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=86352aa42570636b3aa8a48d5d7ced95, ASSIGN}] 2023-02-11 09:55:47,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=86352aa42570636b3aa8a48d5d7ced95, ASSIGN 2023-02-11 09:55:47,050 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=86352aa42570636b3aa8a48d5d7ced95, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41949,1676109241776; forceNewPlan=false, retain=false 2023-02-11 09:55:47,200 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 09:55:47,201 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=86352aa42570636b3aa8a48d5d7ced95, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,201 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109347201"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109347201"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109347201"}]},"ts":"1676109347201"} 2023-02-11 09:55:47,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 86352aa42570636b3aa8a48d5d7ced95, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:47,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 86352aa42570636b3aa8a48d5d7ced95, NAME => 'testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.', STARTKEY => '', ENDKEY => ''} 2023-02-11 09:55:47,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:47,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:47,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,357 INFO [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,359 DEBUG [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1 2023-02-11 09:55:47,359 DEBUG [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1 2023-02-11 09:55:47,359 INFO [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86352aa42570636b3aa8a48d5d7ced95 columnFamilyName cf1 2023-02-11 09:55:47,360 INFO [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] regionserver.HStore(310): Store=86352aa42570636b3aa8a48d5d7ced95/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:47,360 INFO [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,361 DEBUG [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2 2023-02-11 09:55:47,361 DEBUG [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2 2023-02-11 09:55:47,361 INFO [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86352aa42570636b3aa8a48d5d7ced95 columnFamilyName cf2 2023-02-11 09:55:47,361 INFO [StoreOpener-86352aa42570636b3aa8a48d5d7ced95-1] regionserver.HStore(310): Store=86352aa42570636b3aa8a48d5d7ced95/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:47,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:47,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:47,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 09:55:47,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 86352aa42570636b3aa8a48d5d7ced95; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70228861, jitterRate=0.046491578221321106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:47,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 86352aa42570636b3aa8a48d5d7ced95: Running coprocessor pre-open hook at 1676109347357Writing region info on filesystem at 1676109347357Initializing all the Stores at 1676109347357Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109347357Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109347357Cleaning up temporary data from old regions at 1676109347363 (+6 ms)Cleaning up detritus from prior splits at 1676109347363Running coprocessor post-open hooks at 1676109347366 (+3 ms)Region opened successfully at 1676109347366 2023-02-11 09:55:47,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95., pid=143, masterSystemTime=1676109347354 2023-02-11 09:55:47,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,368 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=86352aa42570636b3aa8a48d5d7ced95, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,368 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109347368"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109347368"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109347368"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109347368"}]},"ts":"1676109347368"} 2023-02-11 09:55:47,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-11 09:55:47,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 86352aa42570636b3aa8a48d5d7ced95, server=jenkins-hbase4.apache.org,41949,1676109241776 in 167 msec 2023-02-11 09:55:47,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-11 09:55:47,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=86352aa42570636b3aa8a48d5d7ced95, ASSIGN in 322 msec 2023-02-11 09:55:47,372 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 09:55:47,372 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109347372"}]},"ts":"1676109347372"} 2023-02-11 09:55:47,373 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-11 09:55:47,375 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 09:55:47,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 373 msec 2023-02-11 09:55:47,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:47,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 09:55:47,511 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 09:55:47,511 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,511 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 09:55:47,511 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 09:55:47,511 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,511 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,512 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,512 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,512 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,512 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,513 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:47,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:47,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:47,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:47,514 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:47,514 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:47,514 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. started... 2023-02-11 09:55:47,515 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 86352aa42570636b3aa8a48d5d7ced95 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 09:55:47,517 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,518 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,518 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:47,518 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,518 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,518 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:47,523 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/894b1157207c4b99b1d9e756b034f3d9 2023-02-11 09:55:47,538 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/fa2db6f9c18f41c8a8734b75d4104857 2023-02-11 09:55:47,542 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/894b1157207c4b99b1d9e756b034f3d9 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9 2023-02-11 09:55:47,545 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:47,546 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/fa2db6f9c18f41c8a8734b75d4104857 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857 2023-02-11 09:55:47,549 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:47,550 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 35ms, sequenceid=6, compaction requested=false 2023-02-11 09:55:47,550 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-11 09:55:47,550 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 86352aa42570636b3aa8a48d5d7ced95: 2023-02-11 09:55:47,550 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,550 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:47,550 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:47,550 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,550 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:47,550 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:47,552 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,552 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:47,552 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:47,552 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:47,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 09:55:47,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fc1bbfd[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:47,554 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 09:55:47,554 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:47,555 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,555 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:47,556 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:47,557 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,557 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:47,557 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:47,558 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:47,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:47,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:47,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 09:55:47,564 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 09:55:47,564 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:47,564 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:47,564 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 09:55:47,564 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:47,565 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:47,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:47,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,565 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:47,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:47,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:47,566 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:47,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:47,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:47,566 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:47,567 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,568 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:47,568 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:47,569 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,571 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:47,572 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:47,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 09:55:47,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:47,572 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:47,573 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:47,573 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,573 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,573 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,573 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:47,573 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:47,673 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:47,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:47,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:47,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:47,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 09:55:47,684 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 09:55:47,684 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,684 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 09:55:47,684 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 09:55:47,685 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,685 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,686 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,686 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,686 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,686 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:47,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,686 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:47,686 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:47,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:47,687 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:47,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:47,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:47,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:47,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:47,688 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:47,689 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,689 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,690 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,690 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:47,690 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:47,690 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:47,692 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:47,692 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:47,692 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:47,692 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. started... 2023-02-11 09:55:47,692 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 86352aa42570636b3aa8a48d5d7ced95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:47,706 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/4d1d9e9fba3b438f9e788dfd332cff31 2023-02-11 09:55:48,117 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/eeea0eda94c645038e154c4075c6844b 2023-02-11 09:55:48,121 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/4d1d9e9fba3b438f9e788dfd332cff31 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31 2023-02-11 09:55:48,124 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:48,125 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/eeea0eda94c645038e154c4075c6844b as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b 2023-02-11 09:55:48,128 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 09:55:48,129 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 437ms, sequenceid=19, compaction requested=false 2023-02-11 09:55:48,129 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 86352aa42570636b3aa8a48d5d7ced95: 2023-02-11 09:55:48,129 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,129 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:48,129 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:48,129 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,129 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,129 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,130 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,130 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,131 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,131 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 09:55:48,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b930827[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:48,132 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 09:55:48,133 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,136 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:48,136 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,136 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:48,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,137 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,137 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,137 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:48,137 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,137 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,137 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,139 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:48,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 09:55:48,142 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 09:55:48,142 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:48,142 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:48,142 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 09:55:48,142 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:48,145 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,145 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,145 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,145 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,146 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,146 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,147 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,147 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,148 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 09:55:48,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,150 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:48,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:48,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,150 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,151 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,251 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:48,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:48,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:48,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:48,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 09:55:48,262 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 09:55:48,262 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,262 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 09:55:48,262 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 09:55:48,263 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,263 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,264 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,264 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,264 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,264 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:48,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,265 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,265 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,266 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:48,266 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,266 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. started... 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,266 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,266 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 86352aa42570636b3aa8a48d5d7ced95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:48,272 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,274 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/b06bb3caca244073a639516aebefa898 2023-02-11 09:55:48,284 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/93bf144b1e5341da9fc718ed0f9327c6 2023-02-11 09:55:48,287 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/b06bb3caca244073a639516aebefa898 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898 2023-02-11 09:55:48,290 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:48,291 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/93bf144b1e5341da9fc718ed0f9327c6 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6 2023-02-11 09:55:48,293 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 09:55:48,294 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 28ms, sequenceid=32, compaction requested=true 2023-02-11 09:55:48,294 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 86352aa42570636b3aa8a48d5d7ced95: 2023-02-11 09:55:48,294 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:48,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:48,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,294 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,296 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 09:55:48,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66eed783[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:48,298 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 09:55:48,298 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,299 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,299 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:48,299 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,300 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,299 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,300 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:48,299 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,300 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,300 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:48,300 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:48,301 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,301 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,301 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,301 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,302 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,302 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,302 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,302 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,302 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,302 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:48,305 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 09:55:48,305 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 09:55:48,305 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:48,305 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:48,305 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 09:55:48,305 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:48,307 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,307 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,308 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,308 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,310 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,310 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,310 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,313 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 09:55:48,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:48,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,313 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,314 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:48,314 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,314 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,314 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:48,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,314 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,414 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:48,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:48,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:48,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:48,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 09:55:48,426 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 09:55:48,426 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,426 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 09:55:48,426 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 09:55:48,426 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,426 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,427 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,427 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,427 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,427 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,428 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,428 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,428 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:48,429 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:48,429 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,430 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. started... 2023-02-11 09:55:48,430 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 86352aa42570636b3aa8a48d5d7ced95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:48,431 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,431 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,432 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,432 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,432 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,432 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,437 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/5cafc0df31764464a7465dc0acd8ad5a 2023-02-11 09:55:48,450 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/b99e569088824579b6d126867ded60ea 2023-02-11 09:55:48,453 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/5cafc0df31764464a7465dc0acd8ad5a as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a 2023-02-11 09:55:48,455 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:48,456 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/b99e569088824579b6d126867ded60ea as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea 2023-02-11 09:55:48,459 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 09:55:48,460 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 30ms, sequenceid=45, compaction requested=true 2023-02-11 09:55:48,460 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 86352aa42570636b3aa8a48d5d7ced95: 2023-02-11 09:55:48,460 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,460 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:48,460 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:48,460 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,460 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,460 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,461 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,461 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,461 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,461 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 09:55:48,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1395915[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:48,463 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 09:55:48,463 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:48,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,464 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:48,466 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,466 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,466 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,468 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,468 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,468 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,468 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,468 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,468 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,468 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:48,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 09:55:48,472 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 09:55:48,472 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:48,472 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:48,472 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 09:55:48,472 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:48,474 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,474 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,474 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,474 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,474 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,475 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,475 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,477 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,478 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,478 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 09:55:48,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:48,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,483 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,483 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,484 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:48,484 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,485 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,485 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:48,485 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,485 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,485 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,485 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,585 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:48,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:48,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:48,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:48,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 09:55:48,596 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 09:55:48,596 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,596 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 09:55:48,596 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 09:55:48,597 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,597 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,598 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,598 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,598 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,598 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,598 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,598 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:48,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,599 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,599 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,599 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,599 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,600 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,600 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,600 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. started... 2023-02-11 09:55:48,600 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 86352aa42570636b3aa8a48d5d7ced95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:48,601 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,608 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/8af01ea667234ffca4a4ec21d984bd5d 2023-02-11 09:55:48,618 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/8b564d7a18fa4facb1a60e5e4a956b38 2023-02-11 09:55:48,621 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/8af01ea667234ffca4a4ec21d984bd5d as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d 2023-02-11 09:55:48,624 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:48,625 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/8b564d7a18fa4facb1a60e5e4a956b38 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38 2023-02-11 09:55:48,627 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 09:55:48,628 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 28ms, sequenceid=58, compaction requested=true 2023-02-11 09:55:48,628 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 86352aa42570636b3aa8a48d5d7ced95: 2023-02-11 09:55:48,628 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,628 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:48,628 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:48,628 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,628 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,628 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,629 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,629 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,630 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,630 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 09:55:48,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a5049c[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:48,631 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 09:55:48,631 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,633 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:48,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:48,634 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,634 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,634 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,635 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:48,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 09:55:48,639 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 09:55:48,639 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:48,639 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:48,639 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 09:55:48,639 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:48,640 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,640 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,640 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,640 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,641 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,641 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,642 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,643 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,643 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,643 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 09:55:48,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:48,646 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,646 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,647 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,647 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:48,647 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,647 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,647 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,647 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,747 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:48,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:48,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 09:55:48,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 09:55:48,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 09:55:48,758 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 09:55:48,758 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,759 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 09:55:48,759 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 09:55:48,759 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,759 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,760 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,760 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,760 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,760 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,761 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,761 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,762 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,762 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40095,1676109241701' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 09:55:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 09:55:48,763 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 09:55:48,763 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 09:55:48,763 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,763 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 09:55:48,763 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. started... 2023-02-11 09:55:48,763 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 09:55:48,763 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 09:55:48,763 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 09:55:48,763 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 86352aa42570636b3aa8a48d5d7ced95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 09:55:48,763 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 09:55:48,763 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,763 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34799,1676109241745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,764 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,765 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,765 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,765 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,766 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,766 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,772 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/905090951cfe48b1b74f2db59613a5c8 2023-02-11 09:55:48,783 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/1004877015ea4f59afb10c47e3d67739 2023-02-11 09:55:48,787 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/905090951cfe48b1b74f2db59613a5c8 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8 2023-02-11 09:55:48,790 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:48,790 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/1004877015ea4f59afb10c47e3d67739 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739 2023-02-11 09:55:48,793 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 09:55:48,794 INFO [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 31ms, sequenceid=71, compaction requested=true 2023-02-11 09:55:48,794 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 86352aa42570636b3aa8a48d5d7ced95: 2023-02-11 09:55:48,794 DEBUG [rs(jenkins-hbase4.apache.org,41949,1676109241776)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 09:55:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 09:55:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 09:55:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 09:55:48,795 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,795 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,796 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,796 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 09:55:48,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41949,1676109241776' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 09:55:48,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3044e6ff[Count = 0] remaining members to acquire global barrier 2023-02-11 09:55:48,798 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 09:55:48,798 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,799 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 09:55:48,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40095,1676109241701' in zk 2023-02-11 09:55:48,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41949,1676109241776' in zk 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 09:55:48,799 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34799,1676109241745' in zk 2023-02-11 09:55:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,40095,1676109241701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,805 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,41949,1676109241776' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 09:55:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,34799,1676109241745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 09:55:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41949,1676109241776': 2023-02-11 09:55:48,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41949,1676109241776' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 09:55:48,808 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 09:55:48,808 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 09:55:48,808 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 09:55:48,808 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 09:55:48,808 INFO [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 09:55:48,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,811 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 09:55:48,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,811 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,811 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 09:55:48,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,812 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 09:55:48,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,813 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,813 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,813 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 09:55:48,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 09:55:48,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,818 DEBUG [(jenkins-hbase4.apache.org,43041,1676109240552)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,818 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-02-11 09:55:48,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 09:55:48,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 09:55:48,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 09:55:48,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 09:55:48,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 09:55:48,818 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 09:55:48,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,819 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 09:55:48,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 09:55:48,819 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 09:55:48,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 09:55:48,819 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 09:55:48,919 DEBUG [Listener at localhost/34597] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 09:55:48,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 09:55:48,927 DEBUG [Listener at localhost/34597] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,927 DEBUG [Listener at localhost/34597] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=86352aa42570636b3aa8a48d5d7ced95, daughterA=5577b031e527adbbf863b313b7151f12, daughterB=d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:48,929 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=86352aa42570636b3aa8a48d5d7ced95, daughterA=5577b031e527adbbf863b313b7151f12, daughterB=d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:48,929 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=86352aa42570636b3aa8a48d5d7ced95, daughterA=5577b031e527adbbf863b313b7151f12, daughterB=d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:48,929 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=86352aa42570636b3aa8a48d5d7ced95, daughterA=5577b031e527adbbf863b313b7151f12, daughterB=d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:48,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=86352aa42570636b3aa8a48d5d7ced95, UNASSIGN}] 2023-02-11 09:55:48,931 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=86352aa42570636b3aa8a48d5d7ced95, UNASSIGN 2023-02-11 09:55:48,931 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=86352aa42570636b3aa8a48d5d7ced95, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:48,931 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109348931"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109348931"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109348931"}]},"ts":"1676109348931"} 2023-02-11 09:55:48,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 86352aa42570636b3aa8a48d5d7ced95, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:49,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 86352aa42570636b3aa8a48d5d7ced95, disabling compactions & flushes 2023-02-11 09:55:49,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:49,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:49,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. after waiting 0 ms 2023-02-11 09:55:49,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:49,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 86352aa42570636b3aa8a48d5d7ced95 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 09:55:49,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/1ed6f0b4f9274097a8c5ec29f13625a5 2023-02-11 09:55:49,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/f6b99e272ce34c6cac42e0ca82e2305d 2023-02-11 09:55:49,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf1/1ed6f0b4f9274097a8c5ec29f13625a5 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5 2023-02-11 09:55:49,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:49,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/.tmp/cf2/f6b99e272ce34c6cac42e0ca82e2305d as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d 2023-02-11 09:55:49,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 09:55:49,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 28ms, sequenceid=82, compaction requested=true 2023-02-11 09:55:49,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 09:55:49,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:49,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. 2023-02-11 09:55:49,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 86352aa42570636b3aa8a48d5d7ced95: Waiting for close lock at 1676109349084Running coprocessor pre-close hooks at 1676109349084Disabling compacts and flushes for region at 1676109349084Disabling writes for close at 1676109349084Obtaining lock to block concurrent updates at 1676109349084Preparing flush snapshotting stores in 86352aa42570636b3aa8a48d5d7ced95 at 1676109349084Finished memstore snapshotting testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676109349084Flushing stores of testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. at 1676109349085 (+1 ms)Flushing 86352aa42570636b3aa8a48d5d7ced95/cf1: creating writer at 1676109349086 (+1 ms)Flushing 86352aa42570636b3aa8a48d5d7ced95/cf1: appending metadata at 1676109349087 (+1 ms)Flushing 86352aa42570636b3aa8a48d5d7ced95/cf1: closing flushed file at 1676109349087Flushing 86352aa42570636b3aa8a48d5d7ced95/cf2: creating writer at 1676109349096 (+9 ms)Flushing 86352aa42570636b3aa8a48d5d7ced95/cf2: appending metadata at 1676109349097 (+1 ms)Flushing 86352aa42570636b3aa8a48d5d7ced95/cf2: closing flushed file at 1676109349097Flushing 86352aa42570636b3aa8a48d5d7ced95/cf1: reopening flushed file at 1676109349106 (+9 ms)Flushing 86352aa42570636b3aa8a48d5d7ced95/cf2: reopening flushed file at 1676109349109 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 86352aa42570636b3aa8a48d5d7ced95 in 28ms, sequenceid=82, compaction requested=true at 1676109349112 (+3 ms)Writing region close event to WAL at 1676109349114 (+2 ms)Running coprocessor post-close hooks at 1676109349117 (+3 ms)Closed at 1676109349117 2023-02-11 09:55:49,118 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,119 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=86352aa42570636b3aa8a48d5d7ced95, regionState=CLOSED 2023-02-11 09:55:49,119 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109349119"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109349119"}]},"ts":"1676109349119"} 2023-02-11 09:55:49,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-11 09:55:49,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 86352aa42570636b3aa8a48d5d7ced95, server=jenkins-hbase4.apache.org,41949,1676109241776 in 188 msec 2023-02-11 09:55:49,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-11 09:55:49,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=86352aa42570636b3aa8a48d5d7ced95, UNASSIGN in 191 msec 2023-02-11 09:55:49,130 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=86352aa42570636b3aa8a48d5d7ced95, threads=14 2023-02-11 09:55:49,130 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,130 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,130 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,130 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,130 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,130 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,130 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,131 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,131 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,131 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,131 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,131 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,131 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,131 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,249 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,272 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,300 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,300 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,300 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,310 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,311 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,316 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,316 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,324 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,326 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,352 ERROR [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid 75b11f66-72e0-4ff6-b627-e436c09781c9) service to localhost/127.0.0.1:41137 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:55:49,643 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,645 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,646 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9 for region: 86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:49,646 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 86352aa42570636b3aa8a48d5d7ced95 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 09:55:49,679 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:55:49,680 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 09:55:49,681 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676109349681"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676109349681"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676109349681"}]},"ts":"1676109349681"} 2023-02-11 09:55:49,681 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109349681"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109349681"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109349681"}]},"ts":"1676109349681"} 2023-02-11 09:55:49,682 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109349681"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109349681"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109349681"}]},"ts":"1676109349681"} 2023-02-11 09:55:49,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=5577b031e527adbbf863b313b7151f12, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=d630c3f6b8ba8f7bebc0d459e90fbc5f, ASSIGN}] 2023-02-11 09:55:49,685 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=d630c3f6b8ba8f7bebc0d459e90fbc5f, ASSIGN 2023-02-11 09:55:49,686 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=d630c3f6b8ba8f7bebc0d459e90fbc5f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41949,1676109241776; forceNewPlan=false, retain=false 2023-02-11 09:55:49,686 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=5577b031e527adbbf863b313b7151f12, ASSIGN 2023-02-11 09:55:49,687 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=5577b031e527adbbf863b313b7151f12, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41949,1676109241776; forceNewPlan=false, retain=false 2023-02-11 09:55:49,836 INFO [jenkins-hbase4:43041] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 09:55:49,838 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=d630c3f6b8ba8f7bebc0d459e90fbc5f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:49,838 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109349838"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109349838"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109349838"}]},"ts":"1676109349838"} 2023-02-11 09:55:49,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure d630c3f6b8ba8f7bebc0d459e90fbc5f, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:49,841 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=5577b031e527adbbf863b313b7151f12, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:49,842 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109349841"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109349841"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109349841"}]},"ts":"1676109349841"} 2023-02-11 09:55:49,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure 5577b031e527adbbf863b313b7151f12, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:49,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:49,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d630c3f6b8ba8f7bebc0d459e90fbc5f, NAME => 'testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 09:55:49,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:49,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:49,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:49,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:49,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:49,995 INFO [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:49,996 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1 2023-02-11 09:55:49,996 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1 2023-02-11 09:55:49,996 INFO [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d630c3f6b8ba8f7bebc0d459e90fbc5f columnFamilyName cf1 2023-02-11 09:55:50,008 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5-top 2023-02-11 09:55:50,011 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31-top 2023-02-11 09:55:50,013 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a-top 2023-02-11 09:55:50,016 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9-top 2023-02-11 09:55:50,019 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d-top 2023-02-11 09:55:50,023 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8-top 2023-02-11 09:55:50,026 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898-top 2023-02-11 09:55:50,026 INFO [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(310): Store=d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:50,026 INFO [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,027 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2 2023-02-11 09:55:50,027 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2 2023-02-11 09:55:50,027 INFO [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d630c3f6b8ba8f7bebc0d459e90fbc5f columnFamilyName cf2 2023-02-11 09:55:50,038 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739-top 2023-02-11 09:55:50,041 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38-top 2023-02-11 09:55:50,044 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6-top 2023-02-11 09:55:50,047 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea-top 2023-02-11 09:55:50,050 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b-top 2023-02-11 09:55:50,053 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d-top 2023-02-11 09:55:50,056 DEBUG [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857-top 2023-02-11 09:55:50,057 INFO [StoreOpener-d630c3f6b8ba8f7bebc0d459e90fbc5f-1] regionserver.HStore(310): Store=d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:50,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:50,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d630c3f6b8ba8f7bebc0d459e90fbc5f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70169060, jitterRate=0.04560047388076782}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:50,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d630c3f6b8ba8f7bebc0d459e90fbc5f: Running coprocessor pre-open hook at 1676109349994Writing region info on filesystem at 1676109349994Initializing all the Stores at 1676109349995 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109349995Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109349995Cleaning up temporary data from old regions at 1676109350059 (+64 ms)Cleaning up detritus from prior splits at 1676109350060 (+1 ms)Running coprocessor post-open hooks at 1676109350062 (+2 ms)Region opened successfully at 1676109350062 2023-02-11 09:55:50,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f., pid=149, masterSystemTime=1676109349991 2023-02-11 09:55:50,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:55:50,064 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:50,064 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:50,064 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,064 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:50,064 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1912): d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2 is initiating minor compaction (all files) 2023-02-11 09:55:50,064 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1912): d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1 is initiating minor compaction (all files) 2023-02-11 09:55:50,064 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2 in testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,064 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1 in testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,065 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d-top] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/.tmp, totalSize=34.9 K 2023-02-11 09:55:50,065 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5-top] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/.tmp, totalSize=34.9 K 2023-02-11 09:55:50,065 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109347506 2023-02-11 09:55:50,066 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676109347506 2023-02-11 09:55:50,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,066 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109347674 2023-02-11 09:55:50,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5577b031e527adbbf863b313b7151f12, NAME => 'testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 09:55:50,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 09:55:50,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 09:55:50,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,067 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=d630c3f6b8ba8f7bebc0d459e90fbc5f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,067 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109348252 2023-02-11 09:55:50,067 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109350067"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109350067"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109350067"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109350067"}]},"ts":"1676109350067"} 2023-02-11 09:55:50,067 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676109347674 2023-02-11 09:55:50,068 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676109348252 2023-02-11 09:55:50,068 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109348415 2023-02-11 09:55:50,068 INFO [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,069 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676109348415 2023-02-11 09:55:50,069 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676109348586 2023-02-11 09:55:50,069 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676109348586 2023-02-11 09:55:50,069 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1 2023-02-11 09:55:50,069 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676109348748 2023-02-11 09:55:50,069 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1 2023-02-11 09:55:50,070 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676109348748 2023-02-11 09:55:50,070 INFO [StoreOpener-5577b031e527adbbf863b313b7151f12-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5577b031e527adbbf863b313b7151f12 columnFamilyName cf1 2023-02-11 09:55:50,070 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109348921 2023-02-11 09:55:50,070 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676109348921 2023-02-11 09:55:50,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-11 09:55:50,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure d630c3f6b8ba8f7bebc0d459e90fbc5f, server=jenkins-hbase4.apache.org,41949,1676109241776 in 229 msec 2023-02-11 09:55:50,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=d630c3f6b8ba8f7bebc0d459e90fbc5f, ASSIGN in 388 msec 2023-02-11 09:55:50,083 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] throttle.PressureAwareThroughputController(145): d630c3f6b8ba8f7bebc0d459e90fbc5f#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:50,083 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] throttle.PressureAwareThroughputController(145): d630c3f6b8ba8f7bebc0d459e90fbc5f#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:50,085 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5-bottom 2023-02-11 09:55:50,087 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31-bottom 2023-02-11 09:55:50,099 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a-bottom 2023-02-11 09:55:50,106 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9-bottom 2023-02-11 09:55:50,109 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d-bottom 2023-02-11 09:55:50,110 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/.tmp/cf1/18b13a1842c1482e90ef61b8a001b301 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/18b13a1842c1482e90ef61b8a001b301 2023-02-11 09:55:50,113 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8-bottom 2023-02-11 09:55:50,114 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/.tmp/cf2/e2f49a7644424d6488bb9ce7a19477ba as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/e2f49a7644424d6488bb9ce7a19477ba 2023-02-11 09:55:50,115 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898-bottom 2023-02-11 09:55:50,115 INFO [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(310): Store=5577b031e527adbbf863b313b7151f12/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:50,116 INFO [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,116 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1 of d630c3f6b8ba8f7bebc0d459e90fbc5f into 18b13a1842c1482e90ef61b8a001b301(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:50,116 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d630c3f6b8ba8f7bebc0d459e90fbc5f: 2023-02-11 09:55:50,116 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f., storeName=d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1, priority=9, startTime=1676109350063; duration=0sec 2023-02-11 09:55:50,116 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:50,116 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2 2023-02-11 09:55:50,116 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2 2023-02-11 09:55:50,117 INFO [StoreOpener-5577b031e527adbbf863b313b7151f12-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5577b031e527adbbf863b313b7151f12 columnFamilyName cf2 2023-02-11 09:55:50,118 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2 of d630c3f6b8ba8f7bebc0d459e90fbc5f into e2f49a7644424d6488bb9ce7a19477ba(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:50,118 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d630c3f6b8ba8f7bebc0d459e90fbc5f: 2023-02-11 09:55:50,118 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f., storeName=d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2, priority=9, startTime=1676109350064; duration=0sec 2023-02-11 09:55:50,118 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:50,126 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739-bottom 2023-02-11 09:55:50,128 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38-bottom 2023-02-11 09:55:50,131 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6-bottom 2023-02-11 09:55:50,135 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea-bottom 2023-02-11 09:55:50,138 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b-bottom 2023-02-11 09:55:50,141 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d-bottom 2023-02-11 09:55:50,143 DEBUG [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(539): loaded hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857-bottom 2023-02-11 09:55:50,143 INFO [StoreOpener-5577b031e527adbbf863b313b7151f12-1] regionserver.HStore(310): Store=5577b031e527adbbf863b313b7151f12/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 09:55:50,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 09:55:50,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5577b031e527adbbf863b313b7151f12; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60225795, jitterRate=-0.1025657206773758}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 09:55:50,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5577b031e527adbbf863b313b7151f12: Running coprocessor pre-open hook at 1676109350067Writing region info on filesystem at 1676109350067Initializing all the Stores at 1676109350068 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109350068Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676109350068Cleaning up temporary data from old regions at 1676109350145 (+77 ms)Cleaning up detritus from prior splits at 1676109350145Running coprocessor post-open hooks at 1676109350147 (+2 ms)Region opened successfully at 1676109350147 2023-02-11 09:55:50,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12., pid=150, masterSystemTime=1676109349991 2023-02-11 09:55:50,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:50,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 09:55:50,149 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:50,149 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 09:55:50,149 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,149 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,149 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1912): 5577b031e527adbbf863b313b7151f12/cf1 is initiating minor compaction (all files) 2023-02-11 09:55:50,149 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1912): 5577b031e527adbbf863b313b7151f12/cf2 is initiating minor compaction (all files) 2023-02-11 09:55:50,149 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5577b031e527adbbf863b313b7151f12/cf1 in testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,149 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5577b031e527adbbf863b313b7151f12/cf2 in testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,149 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/.tmp, totalSize=34.9 K 2023-02-11 09:55:50,149 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d-bottom] into tmpdir=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/.tmp, totalSize=34.9 K 2023-02-11 09:55:50,150 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109347506 2023-02-11 09:55:50,150 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676109347506 2023-02-11 09:55:50,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,150 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109347674 2023-02-11 09:55:50,150 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676109347674 2023-02-11 09:55:50,151 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=5577b031e527adbbf863b313b7151f12, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,151 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109348252 2023-02-11 09:55:50,151 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109350151"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676109350151"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676109350151"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676109350151"}]},"ts":"1676109350151"} 2023-02-11 09:55:50,151 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676109348252 2023-02-11 09:55:50,151 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109348415 2023-02-11 09:55:50,151 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676109348415 2023-02-11 09:55:50,152 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109348586 2023-02-11 09:55:50,152 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676109348586 2023-02-11 09:55:50,152 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109348748 2023-02-11 09:55:50,152 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting 1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676109348748 2023-02-11 09:55:50,152 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] compactions.Compactor(207): Compacting 1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109348921 2023-02-11 09:55:50,153 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] compactions.Compactor(207): Compacting f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676109348921 2023-02-11 09:55:50,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-11 09:55:50,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure 5577b031e527adbbf863b313b7151f12, server=jenkins-hbase4.apache.org,41949,1676109241776 in 310 msec 2023-02-11 09:55:50,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-11 09:55:50,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=5577b031e527adbbf863b313b7151f12, ASSIGN in 470 msec 2023-02-11 09:55:50,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=86352aa42570636b3aa8a48d5d7ced95, daughterA=5577b031e527adbbf863b313b7151f12, daughterB=d630c3f6b8ba8f7bebc0d459e90fbc5f in 1.2290 sec 2023-02-11 09:55:50,162 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5577b031e527adbbf863b313b7151f12#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:50,162 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] throttle.PressureAwareThroughputController(145): 5577b031e527adbbf863b313b7151f12#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 09:55:50,171 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/.tmp/cf2/ceb47970be2f450d874495aca8942143 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/ceb47970be2f450d874495aca8942143 2023-02-11 09:55:50,171 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/.tmp/cf1/19ee13eadf334ef5b3111c686d04faee as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/19ee13eadf334ef5b3111c686d04faee 2023-02-11 09:55:50,175 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5577b031e527adbbf863b313b7151f12/cf1 of 5577b031e527adbbf863b313b7151f12 into 19ee13eadf334ef5b3111c686d04faee(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:50,175 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5577b031e527adbbf863b313b7151f12/cf2 of 5577b031e527adbbf863b313b7151f12 into ceb47970be2f450d874495aca8942143(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 09:55:50,175 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5577b031e527adbbf863b313b7151f12: 2023-02-11 09:55:50,175 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5577b031e527adbbf863b313b7151f12: 2023-02-11 09:55:50,175 INFO [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12., storeName=5577b031e527adbbf863b313b7151f12/cf1, priority=9, startTime=1676109350148; duration=0sec 2023-02-11 09:55:50,175 INFO [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12., storeName=5577b031e527adbbf863b313b7151f12/cf2, priority=9, startTime=1676109350149; duration=0sec 2023-02-11 09:55:50,175 DEBUG [RS:2;jenkins-hbase4:41949-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:50,175 DEBUG [RS:2;jenkins-hbase4:41949-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 09:55:50,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41949] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:52822 deadline: 1676109410179, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95. is not online on jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,289 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 09:55:50,293 INFO [Listener at localhost/34597] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 09:55:50,294 DEBUG [Listener at localhost/34597] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 09:55:50,304 WARN [Listener at localhost/34597] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 09:55:50,305 INFO [Listener at localhost/34597] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-11 09:55:50,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-11 09:55:50,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-11 09:55:50,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 09:55:50,308 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109350308"}]},"ts":"1676109350308"} 2023-02-11 09:55:50,310 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-11 09:55:50,311 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-11 09:55:50,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=5577b031e527adbbf863b313b7151f12, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=d630c3f6b8ba8f7bebc0d459e90fbc5f, UNASSIGN}] 2023-02-11 09:55:50,312 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=d630c3f6b8ba8f7bebc0d459e90fbc5f, UNASSIGN 2023-02-11 09:55:50,313 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=d630c3f6b8ba8f7bebc0d459e90fbc5f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,313 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109350313"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109350313"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109350313"}]},"ts":"1676109350313"} 2023-02-11 09:55:50,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure d630c3f6b8ba8f7bebc0d459e90fbc5f, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:50,316 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=5577b031e527adbbf863b313b7151f12, UNASSIGN 2023-02-11 09:55:50,317 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=5577b031e527adbbf863b313b7151f12, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109350316"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676109350316"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676109350316"}]},"ts":"1676109350316"} 2023-02-11 09:55:50,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 5577b031e527adbbf863b313b7151f12, server=jenkins-hbase4.apache.org,41949,1676109241776}] 2023-02-11 09:55:50,370 ERROR [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid 6b5fb81e-4ae9-4619-8579-f3d9aa9248cc) service to localhost/127.0.0.1:41137 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 09:55:50,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 09:55:50,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d630c3f6b8ba8f7bebc0d459e90fbc5f, disabling compactions & flushes 2023-02-11 09:55:50,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. after waiting 0 ms 2023-02-11 09:55:50,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,467 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5-top] to archive 2023-02-11 09:55:50,467 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:50,469 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,469 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,470 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,471 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,472 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,473 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,474 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,475 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739-top, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d-top] to archive 2023-02-11 09:55:50,475 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:50,476 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,477 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,478 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,479 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,480 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,481 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,481 DEBUG [StoreCloser-testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:55:50,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:50,485 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f. 2023-02-11 09:55:50,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d630c3f6b8ba8f7bebc0d459e90fbc5f: Waiting for close lock at 1676109350466Running coprocessor pre-close hooks at 1676109350466Disabling compacts and flushes for region at 1676109350466Disabling writes for close at 1676109350466Writing region close event to WAL at 1676109350482 (+16 ms)Running coprocessor post-close hooks at 1676109350484 (+2 ms)Closed at 1676109350485 (+1 ms) 2023-02-11 09:55:50,486 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,486 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,486 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5577b031e527adbbf863b313b7151f12, disabling compactions & flushes 2023-02-11 09:55:50,486 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,486 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,486 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. after waiting 0 ms 2023-02-11 09:55:50,486 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=d630c3f6b8ba8f7bebc0d459e90fbc5f, regionState=CLOSED 2023-02-11 09:55:50,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109350486"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109350486"}]},"ts":"1676109350486"} 2023-02-11 09:55:50,487 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5-bottom] to archive 2023-02-11 09:55:50,488 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:50,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-11 09:55:50,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure d630c3f6b8ba8f7bebc0d459e90fbc5f, server=jenkins-hbase4.apache.org,41949,1676109241776 in 173 msec 2023-02-11 09:55:50,489 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/894b1157207c4b99b1d9e756b034f3d9.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=d630c3f6b8ba8f7bebc0d459e90fbc5f, UNASSIGN in 177 msec 2023-02-11 09:55:50,490 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/4d1d9e9fba3b438f9e788dfd332cff31.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,491 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/b06bb3caca244073a639516aebefa898.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,492 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/5cafc0df31764464a7465dc0acd8ad5a.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,493 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/8af01ea667234ffca4a4ec21d984bd5d.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,494 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/905090951cfe48b1b74f2db59613a5c8.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,495 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/1ed6f0b4f9274097a8c5ec29f13625a5.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,497 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739-bottom, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95->hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d-bottom] to archive 2023-02-11 09:55:50,497 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 09:55:50,498 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/fa2db6f9c18f41c8a8734b75d4104857.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,499 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/eeea0eda94c645038e154c4075c6844b.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,500 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/93bf144b1e5341da9fc718ed0f9327c6.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,501 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/b99e569088824579b6d126867ded60ea.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,502 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/8b564d7a18fa4facb1a60e5e4a956b38.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,502 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/1004877015ea4f59afb10c47e3d67739.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,503 DEBUG [StoreCloser-testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/f6b99e272ce34c6cac42e0ca82e2305d.86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,506 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 09:55:50,506 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:50,506 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12. 2023-02-11 09:55:50,506 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5577b031e527adbbf863b313b7151f12: Waiting for close lock at 1676109350486Running coprocessor pre-close hooks at 1676109350486Disabling compacts and flushes for region at 1676109350486Disabling writes for close at 1676109350486Writing region close event to WAL at 1676109350504 (+18 ms)Running coprocessor post-close hooks at 1676109350506 (+2 ms)Closed at 1676109350506 2023-02-11 09:55:50,507 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,508 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=5577b031e527adbbf863b313b7151f12, regionState=CLOSED 2023-02-11 09:55:50,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676109350508"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676109350508"}]},"ts":"1676109350508"} 2023-02-11 09:55:50,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-11 09:55:50,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 5577b031e527adbbf863b313b7151f12, server=jenkins-hbase4.apache.org,41949,1676109241776 in 192 msec 2023-02-11 09:55:50,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-11 09:55:50,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=5577b031e527adbbf863b313b7151f12, UNASSIGN in 198 msec 2023-02-11 09:55:50,512 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676109350512"}]},"ts":"1676109350512"} 2023-02-11 09:55:50,513 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-11 09:55:50,514 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-11 09:55:50,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 210 msec 2023-02-11 09:55:50,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 09:55:50,610 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-11 09:55:50,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-11 09:55:50,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 09:55:50,613 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 09:55:50,613 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 09:55:50,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 09:55:50,617 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,617 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,617 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,618 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/recovered.edits] 2023-02-11 09:55:50,618 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/recovered.edits] 2023-02-11 09:55:50,619 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2, FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/recovered.edits] 2023-02-11 09:55:50,621 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/18b13a1842c1482e90ef61b8a001b301 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf1/18b13a1842c1482e90ef61b8a001b301 2023-02-11 09:55:50,621 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/19ee13eadf334ef5b3111c686d04faee to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf1/19ee13eadf334ef5b3111c686d04faee 2023-02-11 09:55:50,622 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/1ed6f0b4f9274097a8c5ec29f13625a5 2023-02-11 09:55:50,623 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/e2f49a7644424d6488bb9ce7a19477ba to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/cf2/e2f49a7644424d6488bb9ce7a19477ba 2023-02-11 09:55:50,623 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/4d1d9e9fba3b438f9e788dfd332cff31 2023-02-11 09:55:50,623 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/ceb47970be2f450d874495aca8942143 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/cf2/ceb47970be2f450d874495aca8942143 2023-02-11 09:55:50,624 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/5cafc0df31764464a7465dc0acd8ad5a 2023-02-11 09:55:50,626 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f/recovered.edits/90.seqid 2023-02-11 09:55:50,626 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/894b1157207c4b99b1d9e756b034f3d9 2023-02-11 09:55:50,626 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/recovered.edits/90.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12/recovered.edits/90.seqid 2023-02-11 09:55:50,626 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/d630c3f6b8ba8f7bebc0d459e90fbc5f 2023-02-11 09:55:50,626 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/5577b031e527adbbf863b313b7151f12 2023-02-11 09:55:50,627 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/8af01ea667234ffca4a4ec21d984bd5d 2023-02-11 09:55:50,628 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/905090951cfe48b1b74f2db59613a5c8 2023-02-11 09:55:50,629 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf1/b06bb3caca244073a639516aebefa898 2023-02-11 09:55:50,631 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/1004877015ea4f59afb10c47e3d67739 2023-02-11 09:55:50,632 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/8b564d7a18fa4facb1a60e5e4a956b38 2023-02-11 09:55:50,633 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/93bf144b1e5341da9fc718ed0f9327c6 2023-02-11 09:55:50,634 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/b99e569088824579b6d126867ded60ea 2023-02-11 09:55:50,634 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/eeea0eda94c645038e154c4075c6844b 2023-02-11 09:55:50,635 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/f6b99e272ce34c6cac42e0ca82e2305d 2023-02-11 09:55:50,636 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857 to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/cf2/fa2db6f9c18f41c8a8734b75d4104857 2023-02-11 09:55:50,638 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/recovered.edits/85.seqid to hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/archive/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95/recovered.edits/85.seqid 2023-02-11 09:55:50,638 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/.tmp/data/default/testSplitTableRegion/86352aa42570636b3aa8a48d5d7ced95 2023-02-11 09:55:50,638 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 09:55:50,640 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 09:55:50,641 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-11 09:55:50,643 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-11 09:55:50,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 09:55:50,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-11 09:55:50,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109350644"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:50,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109350644"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:50,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676109350644"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:50,645 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 09:55:50,645 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 86352aa42570636b3aa8a48d5d7ced95, NAME => 'testSplitTableRegion,,1676109347002.86352aa42570636b3aa8a48d5d7ced95.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5577b031e527adbbf863b313b7151f12, NAME => 'testSplitTableRegion,,1676109348927.5577b031e527adbbf863b313b7151f12.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => d630c3f6b8ba8f7bebc0d459e90fbc5f, NAME => 'testSplitTableRegion,41,1676109348927.d630c3f6b8ba8f7bebc0d459e90fbc5f.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 09:55:50,645 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-11 09:55:50,645 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676109350645"}]},"ts":"9223372036854775807"} 2023-02-11 09:55:50,646 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-11 09:55:50,649 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 09:55:50,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 38 msec 2023-02-11 09:55:50,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43041] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 09:55:50,714 INFO [Listener at localhost/34597] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-11 09:55:50,728 INFO [Listener at localhost/34597] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376 (was 375) - Thread LEAK? -, OpenFileDescriptor=833 (was 833), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=103 (was 104), ProcessCount=177 (was 174) - ProcessCount LEAK? -, AvailableMemoryMB=2455 (was 2461) 2023-02-11 09:55:50,728 INFO [Listener at localhost/34597] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-11 09:55:50,728 INFO [Listener at localhost/34597] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 09:55:50,732 DEBUG [Listener at localhost/34597] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37e887de to 127.0.0.1:49784 2023-02-11 09:55:50,732 DEBUG [Listener at localhost/34597] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:50,733 DEBUG [Listener at localhost/34597] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-11 09:55:50,733 DEBUG [Listener at localhost/34597] util.JVMClusterUtil(257): Found active master hash=115348766, stopped=false 2023-02-11 09:55:50,733 INFO [Listener at localhost/34597] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:55:50,735 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 09:55:50,735 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 09:55:50,735 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 09:55:50,735 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 09:55:50,735 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:55:50,735 INFO [Listener at localhost/34597] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 09:55:50,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:55:50,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:55:50,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:55:50,736 DEBUG [Listener at localhost/34597] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x03edf9f1 to 127.0.0.1:49784 2023-02-11 09:55:50,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 09:55:50,736 DEBUG [Listener at localhost/34597] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:50,736 INFO [Listener at localhost/34597] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,40095,1676109241701' ***** 2023-02-11 09:55:50,736 INFO [Listener at localhost/34597] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 09:55:50,736 INFO [Listener at localhost/34597] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34799,1676109241745' ***** 2023-02-11 09:55:50,736 INFO [Listener at localhost/34597] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 09:55:50,736 INFO [Listener at localhost/34597] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41949,1676109241776' ***** 2023-02-11 09:55:50,736 INFO [Listener at localhost/34597] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 09:55:50,737 INFO [RS:0;jenkins-hbase4:40095] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 09:55:50,737 INFO [RS:2;jenkins-hbase4:41949] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 09:55:50,737 INFO [RS:2;jenkins-hbase4:41949] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 09:55:50,737 INFO [RS:1;jenkins-hbase4:34799] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 09:55:50,737 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 09:55:50,737 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 09:55:50,737 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 09:55:50,737 INFO [RS:0;jenkins-hbase4:40095] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 09:55:50,737 INFO [RS:2;jenkins-hbase4:41949] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 09:55:50,737 INFO [RS:1;jenkins-hbase4:34799] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 09:55:50,738 INFO [RS:0;jenkins-hbase4:40095] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 09:55:50,738 INFO [RS:1;jenkins-hbase4:34799] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 09:55:50,738 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:50,738 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,738 DEBUG [RS:0;jenkins-hbase4:40095] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d2905d0 to 127.0.0.1:49784 2023-02-11 09:55:50,738 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(3304): Received CLOSE for 8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:55:50,738 DEBUG [RS:0;jenkins-hbase4:40095] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:50,738 DEBUG [RS:2;jenkins-hbase4:41949] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34bb383e to 127.0.0.1:49784 2023-02-11 09:55:50,738 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,40095,1676109241701; all regions closed. 2023-02-11 09:55:50,738 DEBUG [RS:2;jenkins-hbase4:41949] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:50,738 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41949,1676109241776; all regions closed. 2023-02-11 09:55:50,738 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:50,738 DEBUG [RS:1;jenkins-hbase4:34799] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f9b5be4 to 127.0.0.1:49784 2023-02-11 09:55:50,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8df39b0c7ffdf9ffd52d48b4d3daaa06, disabling compactions & flushes 2023-02-11 09:55:50,738 DEBUG [RS:1;jenkins-hbase4:34799] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:50,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:55:50,739 INFO [RS:1;jenkins-hbase4:34799] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 09:55:50,739 INFO [RS:1;jenkins-hbase4:34799] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 09:55:50,739 INFO [RS:1;jenkins-hbase4:34799] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 09:55:50,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:55:50,739 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-11 09:55:50,891 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 09:55:50,891 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 09:55:50,891 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 09:55:50,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. after waiting 0 ms 2023-02-11 09:55:50,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:55:50,892 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-11 09:55:50,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8df39b0c7ffdf9ffd52d48b4d3daaa06 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-11 09:55:50,893 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1479): Online Regions={8df39b0c7ffdf9ffd52d48b4d3daaa06=hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06., 1588230740=hbase:meta,,1.1588230740} 2023-02-11 09:55:50,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 09:55:50,893 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 09:55:50,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 09:55:50,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 09:55:50,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 09:55:50,893 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-11 09:55:50,893 DEBUG [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1505): Waiting on 1588230740, 8df39b0c7ffdf9ffd52d48b4d3daaa06 2023-02-11 09:55:50,905 DEBUG [RS:2;jenkins-hbase4:41949] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs 2023-02-11 09:55:50,905 DEBUG [RS:0;jenkins-hbase4:40095] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs 2023-02-11 09:55:50,905 INFO [RS:0;jenkins-hbase4:40095] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40095%2C1676109241701:(num 1676109243246) 2023-02-11 09:55:50,905 INFO [RS:2;jenkins-hbase4:41949] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41949%2C1676109241776:(num 1676109243245) 2023-02-11 09:55:50,905 DEBUG [RS:2;jenkins-hbase4:41949] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:50,905 DEBUG [RS:0;jenkins-hbase4:40095] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:50,905 INFO [RS:2;jenkins-hbase4:41949] regionserver.LeaseManager(133): Closed leases 2023-02-11 09:55:50,905 INFO [RS:0;jenkins-hbase4:40095] regionserver.LeaseManager(133): Closed leases 2023-02-11 09:55:50,905 INFO [RS:2;jenkins-hbase4:41949] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 09:55:50,906 INFO [RS:0;jenkins-hbase4:40095] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 09:55:50,906 INFO [RS:2;jenkins-hbase4:41949] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 09:55:50,906 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 09:55:50,906 INFO [RS:2;jenkins-hbase4:41949] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 09:55:50,907 INFO [RS:2;jenkins-hbase4:41949] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 09:55:50,906 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 09:55:50,906 INFO [RS:0;jenkins-hbase4:40095] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 09:55:50,907 INFO [RS:0;jenkins-hbase4:40095] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 09:55:50,907 INFO [RS:0;jenkins-hbase4:40095] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 09:55:50,908 INFO [RS:0;jenkins-hbase4:40095] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40095 2023-02-11 09:55:50,908 INFO [RS:2;jenkins-hbase4:41949] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41949 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40095,1676109241701 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 09:55:50,915 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@483958ad rejected from java.util.concurrent.ThreadPoolExecutor@4c06d572[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 09:55:50,915 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3a39f31a rejected from java.util.concurrent.ThreadPoolExecutor@4c06d572[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:50,915 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,916 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,916 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41949,1676109241776 2023-02-11 09:55:50,916 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4290c7ff rejected from java.util.concurrent.ThreadPoolExecutor@4c06d572[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:50,916 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40095,1676109241701] 2023-02-11 09:55:50,916 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40095,1676109241701; numProcessing=1 2023-02-11 09:55:50,918 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40095,1676109241701 already deleted, retry=false 2023-02-11 09:55:50,918 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40095,1676109241701 expired; onlineServers=2 2023-02-11 09:55:50,918 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41949,1676109241776] 2023-02-11 09:55:50,918 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41949,1676109241776; numProcessing=2 2023-02-11 09:55:50,919 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41949,1676109241776 already deleted, retry=false 2023-02-11 09:55:50,919 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41949,1676109241776 expired; onlineServers=1 2023-02-11 09:55:50,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/.tmp/info/5f78f492f2604041b026a28ecbc749db 2023-02-11 09:55:50,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/.tmp/info/5f78f492f2604041b026a28ecbc749db as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/info/5f78f492f2604041b026a28ecbc749db 2023-02-11 09:55:50,958 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/.tmp/info/9417db04a9234ebda7747b2fcb229f48 2023-02-11 09:55:50,962 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/info/5f78f492f2604041b026a28ecbc749db, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 09:55:50,962 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9417db04a9234ebda7747b2fcb229f48 2023-02-11 09:55:50,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 8df39b0c7ffdf9ffd52d48b4d3daaa06 in 71ms, sequenceid=6, compaction requested=false 2023-02-11 09:55:50,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/namespace/8df39b0c7ffdf9ffd52d48b4d3daaa06/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-11 09:55:50,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:50,968 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:55:50,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8df39b0c7ffdf9ffd52d48b4d3daaa06: Waiting for close lock at 1676109350738Running coprocessor pre-close hooks at 1676109350738Disabling compacts and flushes for region at 1676109350738Disabling writes for close at 1676109350892 (+154 ms)Obtaining lock to block concurrent updates at 1676109350893 (+1 ms)Preparing flush snapshotting stores in 8df39b0c7ffdf9ffd52d48b4d3daaa06 at 1676109350893Finished memstore snapshotting hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676109350893Flushing stores of hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. at 1676109350894 (+1 ms)Flushing 8df39b0c7ffdf9ffd52d48b4d3daaa06/info: creating writer at 1676109350894Flushing 8df39b0c7ffdf9ffd52d48b4d3daaa06/info: appending metadata at 1676109350910 (+16 ms)Flushing 8df39b0c7ffdf9ffd52d48b4d3daaa06/info: closing flushed file at 1676109350910Flushing 8df39b0c7ffdf9ffd52d48b4d3daaa06/info: reopening flushed file at 1676109350958 (+48 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 8df39b0c7ffdf9ffd52d48b4d3daaa06 in 71ms, sequenceid=6, compaction requested=false at 1676109350963 (+5 ms)Writing region close event to WAL at 1676109350964 (+1 ms)Running coprocessor post-close hooks at 1676109350967 (+3 ms)Closed at 1676109350968 (+1 ms) 2023-02-11 09:55:50,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676109243654.8df39b0c7ffdf9ffd52d48b4d3daaa06. 2023-02-11 09:55:50,972 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/.tmp/rep_barrier/5fe2ff7ee85943fd887b8cd9be018fcd 2023-02-11 09:55:50,975 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fe2ff7ee85943fd887b8cd9be018fcd 2023-02-11 09:55:50,986 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/.tmp/table/dcf811f9b3a7406a96be0964f8f8c157 2023-02-11 09:55:50,989 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcf811f9b3a7406a96be0964f8f8c157 2023-02-11 09:55:50,990 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/.tmp/info/9417db04a9234ebda7747b2fcb229f48 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/info/9417db04a9234ebda7747b2fcb229f48 2023-02-11 09:55:50,993 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9417db04a9234ebda7747b2fcb229f48 2023-02-11 09:55:50,993 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/info/9417db04a9234ebda7747b2fcb229f48, entries=59, sequenceid=210, filesize=11.1 K 2023-02-11 09:55:50,993 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/.tmp/rep_barrier/5fe2ff7ee85943fd887b8cd9be018fcd as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/rep_barrier/5fe2ff7ee85943fd887b8cd9be018fcd 2023-02-11 09:55:50,996 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fe2ff7ee85943fd887b8cd9be018fcd 2023-02-11 09:55:50,996 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/rep_barrier/5fe2ff7ee85943fd887b8cd9be018fcd, entries=24, sequenceid=210, filesize=7.5 K 2023-02-11 09:55:50,997 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/.tmp/table/dcf811f9b3a7406a96be0964f8f8c157 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/table/dcf811f9b3a7406a96be0964f8f8c157 2023-02-11 09:55:50,999 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcf811f9b3a7406a96be0964f8f8c157 2023-02-11 09:55:50,999 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/table/dcf811f9b3a7406a96be0964f8f8c157, entries=36, sequenceid=210, filesize=8.0 K 2023-02-11 09:55:51,000 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 107ms, sequenceid=210, compaction requested=false 2023-02-11 09:55:51,004 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-11 09:55:51,004 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 09:55:51,004 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 09:55:51,005 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 09:55:51,005 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676109350893Running coprocessor pre-close hooks at 1676109350893Disabling compacts and flushes for region at 1676109350893Disabling writes for close at 1676109350893Obtaining lock to block concurrent updates at 1676109350893Preparing flush snapshotting stores in 1588230740 at 1676109350893Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676109350893Flushing stores of hbase:meta,,1.1588230740 at 1676109350895 (+2 ms)Flushing 1588230740/info: creating writer at 1676109350895Flushing 1588230740/info: appending metadata at 1676109350912 (+17 ms)Flushing 1588230740/info: closing flushed file at 1676109350912Flushing 1588230740/rep_barrier: creating writer at 1676109350962 (+50 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676109350966 (+4 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676109350966Flushing 1588230740/table: creating writer at 1676109350975 (+9 ms)Flushing 1588230740/table: appending metadata at 1676109350977 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676109350977Flushing 1588230740/info: reopening flushed file at 1676109350990 (+13 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676109350994 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676109350997 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 107ms, sequenceid=210, compaction requested=false at 1676109351000 (+3 ms)Writing region close event to WAL at 1676109351002 (+2 ms)Running coprocessor post-close hooks at 1676109351004 (+2 ms)Closed at 1676109351005 (+1 ms) 2023-02-11 09:55:51,005 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-11 09:55:51,017 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,017 INFO [RS:0;jenkins-hbase4:40095] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,40095,1676109241701; zookeeper connection closed. 2023-02-11 09:55:51,017 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1c7f32c5 rejected from java.util.concurrent.ThreadPoolExecutor@4c06d572[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,017 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:40095-0x10042cd3f510001, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,017 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@59940bdd rejected from java.util.concurrent.ThreadPoolExecutor@4c06d572[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,017 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@34e64ea2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@34e64ea2 2023-02-11 09:55:51,094 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34799,1676109241745; all regions closed. 2023-02-11 09:55:51,098 DEBUG [RS:1;jenkins-hbase4:34799] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs 2023-02-11 09:55:51,098 INFO [RS:1;jenkins-hbase4:34799] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34799%2C1676109241745.meta:.meta(num 1676109243372) 2023-02-11 09:55:51,103 DEBUG [RS:1;jenkins-hbase4:34799] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/oldWALs 2023-02-11 09:55:51,103 INFO [RS:1;jenkins-hbase4:34799] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34799%2C1676109241745:(num 1676109243246) 2023-02-11 09:55:51,103 DEBUG [RS:1;jenkins-hbase4:34799] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:51,103 INFO [RS:1;jenkins-hbase4:34799] regionserver.LeaseManager(133): Closed leases 2023-02-11 09:55:51,103 INFO [RS:1;jenkins-hbase4:34799] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 09:55:51,103 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 09:55:51,104 INFO [RS:1;jenkins-hbase4:34799] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34799 2023-02-11 09:55:51,107 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 09:55:51,107 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34799,1676109241745 2023-02-11 09:55:51,107 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3b9867e5 rejected from java.util.concurrent.ThreadPoolExecutor@3fa7e44e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,108 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34799,1676109241745] 2023-02-11 09:55:51,108 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34799,1676109241745; numProcessing=3 2023-02-11 09:55:51,109 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34799,1676109241745 already deleted, retry=false 2023-02-11 09:55:51,109 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34799,1676109241745 expired; onlineServers=0 2023-02-11 09:55:51,109 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43041,1676109240552' ***** 2023-02-11 09:55:51,109 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-11 09:55:51,110 DEBUG [M:0;jenkins-hbase4:43041] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47dd1f25, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 09:55:51,110 INFO [M:0;jenkins-hbase4:43041] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43041,1676109240552 2023-02-11 09:55:51,110 INFO [M:0;jenkins-hbase4:43041] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43041,1676109240552; all regions closed. 2023-02-11 09:55:51,110 DEBUG [M:0;jenkins-hbase4:43041] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 09:55:51,110 DEBUG [M:0;jenkins-hbase4:43041] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-11 09:55:51,110 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-11 09:55:51,110 DEBUG [M:0;jenkins-hbase4:43041] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-11 09:55:51,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676109242879] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676109242879,5,FailOnTimeoutGroup] 2023-02-11 09:55:51,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676109242879] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676109242879,5,FailOnTimeoutGroup] 2023-02-11 09:55:51,110 INFO [M:0;jenkins-hbase4:43041] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-11 09:55:51,111 INFO [M:0;jenkins-hbase4:43041] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-11 09:55:51,111 INFO [M:0;jenkins-hbase4:43041] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-11 09:55:51,111 DEBUG [M:0;jenkins-hbase4:43041] master.HMaster(1502): Stopping service threads 2023-02-11 09:55:51,111 INFO [M:0;jenkins-hbase4:43041] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-11 09:55:51,111 ERROR [M:0;jenkins-hbase4:43041] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-11 09:55:51,111 INFO [M:0;jenkins-hbase4:43041] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 09:55:51,111 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-11 09:55:51,111 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 09:55:51,111 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 09:55:51,112 DEBUG [M:0;jenkins-hbase4:43041] zookeeper.ZKUtil(398): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-11 09:55:51,112 WARN [M:0;jenkins-hbase4:43041] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-11 09:55:51,112 INFO [M:0;jenkins-hbase4:43041] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 09:55:51,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 09:55:51,112 INFO [M:0;jenkins-hbase4:43041] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-11 09:55:51,112 DEBUG [M:0;jenkins-hbase4:43041] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 09:55:51,112 INFO [M:0;jenkins-hbase4:43041] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:55:51,112 DEBUG [M:0;jenkins-hbase4:43041] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:55:51,112 DEBUG [M:0;jenkins-hbase4:43041] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 09:55:51,112 DEBUG [M:0;jenkins-hbase4:43041] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:55:51,112 INFO [M:0;jenkins-hbase4:43041] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.14 KB heapSize=719.04 KB 2023-02-11 09:55:51,117 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,117 INFO [RS:2;jenkins-hbase4:41949] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41949,1676109241776; zookeeper connection closed. 2023-02-11 09:55:51,117 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@81c7227 rejected from java.util.concurrent.ThreadPoolExecutor@c23e7fc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,118 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@365fe3cb] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@365fe3cb 2023-02-11 09:55:51,118 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:41949-0x10042cd3f510003, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,118 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@13443258 rejected from java.util.concurrent.ThreadPoolExecutor@c23e7fc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,125 INFO [M:0;jenkins-hbase4:43041] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.14 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3a2fa35d03404a39bf5779d54453cbc1 2023-02-11 09:55:51,128 INFO [M:0;jenkins-hbase4:43041] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a2fa35d03404a39bf5779d54453cbc1 2023-02-11 09:55:51,128 DEBUG [M:0;jenkins-hbase4:43041] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3a2fa35d03404a39bf5779d54453cbc1 as hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3a2fa35d03404a39bf5779d54453cbc1 2023-02-11 09:55:51,131 INFO [M:0;jenkins-hbase4:43041] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a2fa35d03404a39bf5779d54453cbc1 2023-02-11 09:55:51,131 INFO [M:0;jenkins-hbase4:43041] regionserver.HStore(1080): Added hdfs://localhost:41137/user/jenkins/test-data/6eff36c6-1717-cc40-94b0-b6e1ad6aea16/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3a2fa35d03404a39bf5779d54453cbc1, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-11 09:55:51,132 INFO [M:0;jenkins-hbase4:43041] regionserver.HRegion(2947): Finished flush of dataSize ~608.14 KB/622734, heapSize ~719.02 KB/736280, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false 2023-02-11 09:55:51,133 INFO [M:0;jenkins-hbase4:43041] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 09:55:51,133 DEBUG [M:0;jenkins-hbase4:43041] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676109351112Disabling compacts and flushes for region at 1676109351112Disabling writes for close at 1676109351112Obtaining lock to block concurrent updates at 1676109351112Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676109351112Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622734, getHeapSize=736280, getOffHeapSize=0, getCellsCount=1499 at 1676109351112Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676109351113 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676109351113Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676109351116 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676109351116Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676109351129 (+13 ms)Finished flush of dataSize ~608.14 KB/622734, heapSize ~719.02 KB/736280, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false at 1676109351132 (+3 ms)Writing region close event to WAL at 1676109351133 (+1 ms)Closed at 1676109351133 2023-02-11 09:55:51,136 INFO [M:0;jenkins-hbase4:43041] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-11 09:55:51,136 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 09:55:51,137 INFO [M:0;jenkins-hbase4:43041] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43041 2023-02-11 09:55:51,140 DEBUG [M:0;jenkins-hbase4:43041] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43041,1676109240552 already deleted, retry=false 2023-02-11 09:55:51,218 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,218 INFO [RS:1;jenkins-hbase4:34799] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34799,1676109241745; zookeeper connection closed. 2023-02-11 09:55:51,218 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@33ca3bcb rejected from java.util.concurrent.ThreadPoolExecutor@3fa7e44e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,218 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): regionserver:34799-0x10042cd3f510002, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,218 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@50ac208c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@50ac208c 2023-02-11 09:55:51,218 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4b10ddb0 rejected from java.util.concurrent.ThreadPoolExecutor@3fa7e44e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,218 INFO [Listener at localhost/34597] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-11 09:55:51,318 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,318 INFO [M:0;jenkins-hbase4:43041] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43041,1676109240552; zookeeper connection closed. 2023-02-11 09:55:51,318 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4b25209a rejected from java.util.concurrent.ThreadPoolExecutor@49bd6a88[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,318 DEBUG [Listener at localhost/34597-EventThread] zookeeper.ZKWatcher(600): master:43041-0x10042cd3f510000, quorum=127.0.0.1:49784, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 09:55:51,318 ERROR [Listener at localhost/34597-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@12b15bbb rejected from java.util.concurrent.ThreadPoolExecutor@49bd6a88[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 09:55:51,320 WARN [Listener at localhost/34597] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 09:55:51,323 INFO [Listener at localhost/34597] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 09:55:51,394 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 09:55:51,441 WARN [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid 6b5fb81e-4ae9-4619-8579-f3d9aa9248cc) service to localhost/127.0.0.1:41137 2023-02-11 09:55:51,443 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data5/current/BP-122337089-172.31.14.131-1676109237390] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 09:55:51,443 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data6/current/BP-122337089-172.31.14.131-1676109237390] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 09:55:51,445 WARN [Listener at localhost/34597] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 09:55:51,449 INFO [Listener at localhost/34597] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 09:55:51,551 WARN [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid 75b11f66-72e0-4ff6-b627-e436c09781c9) service to localhost/127.0.0.1:41137 2023-02-11 09:55:51,551 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data3/current/BP-122337089-172.31.14.131-1676109237390] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 09:55:51,552 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data4/current/BP-122337089-172.31.14.131-1676109237390] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 09:55:51,553 WARN [Listener at localhost/34597] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 09:55:51,555 INFO [Listener at localhost/34597] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 09:55:51,659 WARN [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 09:55:51,659 WARN [BP-122337089-172.31.14.131-1676109237390 heartbeating to localhost/127.0.0.1:41137] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-122337089-172.31.14.131-1676109237390 (Datanode Uuid b979c166-8ceb-4cd3-99a3-1b2b2b1e0014) service to localhost/127.0.0.1:41137 2023-02-11 09:55:51,660 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data1/current/BP-122337089-172.31.14.131-1676109237390] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 09:55:51,660 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d6db5571-256c-250e-1e13-a4c5ff5c48c9/cluster_1c9e427d-f9f0-0c49-72e5-11e687edd8e7/dfs/data/data2/current/BP-122337089-172.31.14.131-1676109237390] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 09:55:51,680 INFO [Listener at localhost/34597] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 09:55:51,870 INFO [Listener at localhost/34597] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-11 09:55:52,098 INFO [Listener at localhost/34597] hbase.HBaseTestingUtility(1293): Minicluster is down